training vs validation accuracy and loss output error
Can anyone help me interpret these graphs.
I have been training some datasets over SENet for image classification (tensorLayer).
I started with cifar10 and got some satisfying results with it I use data augmentation for all my experiments.
Next i tried training it with Caltech256 and the results were pretty unusual Maybe due to less number of images per class(3050)
Finally i used TinyImagenet with enough training samples per class(500) but the results are still looking odd somewhat similar to caltech256 results
Is that supposed to be a problem with the data??
See also questions close to this topic

Multivariate Gaussian distribution
I was going through Andrew Ng's Machine learning course and was a bit confused about the difference between Gaussian distribution and multivariate Gaussian distribution. As per my understanding multivariate Gaussian distribution is for models where we have multiple features like x1,x2...xn but the same thing could be done for original model. I know I am totally confused here.

Predict future values after using polynomial regression in python
I'm currently using TensorFlow and SkLearn to to try to make a model that can predict the amount of sales for a certain product, X, based on the outdoor temperature in celcius.
I took my datasets for the temperature and set it equal to the x variable, and the amount of sales to as a y variable. As seen on the picture below, there is some sort of correlation between the temperature and the amount of sales:
First and foremost, I tried to do linear regression to see how well it'd fit. This is the code for that:
from sklearn.linear_model import LinearRegression model = LinearRegression() model.fit(x_train, y_train) #fit tries to fit the x variable and y variable. #Let's try to plot it out. y_pred = model.predict(x_train) plt.scatter(x_train,y_train) plt.plot(x_train,y_pred,'r') plt.legend(['Predicted Line', 'Observed data']) plt.show()
This resulted in a predicted line that had a pretty poor fit:
A very nice feature from sklearn however is that you can try to predict an value based on a temperature, so if I were to write
model.predict(15)
i'd get the output
array([6949.05567873])
This is exactly what I want, I just wanted to line to fit better so instead I tried polynoimal regression with sklearn by doing following:
from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=8, include_bias=False) #the bias is avoiding the need to intercept x_new = poly.fit_transform(x_train) new_model = LinearRegression() new_model.fit(x_new,y_train) #plotting y_prediction = new_model.predict(x_new) #this actually predicts x...? plt.scatter(x_train,y_train) plt.plot(x_new[:,0], y_prediction, 'r') plt.legend(['Predicted line', 'Observed data']) plt.show()
The line seems to fit better now:
My problem is not that I can't use new_model.predict(x) since it'll result in "ValueError: shapes (1,1) and (8,) not aligned: 1 (dim 1) != 8 (dim 0)". I understand that this is because I'm using a 8degree polynomium, but is there any way for me to predict the yaxsis based on ONE temperature using the polynomial regression model?

How to solve a ResourceExhaustedError error in Python reinforcement learning
I use the python3.6.5,tensorflow,spyder.My GPU is nvidia gt750m.I am implementing a policy gradient with gym's MsPacmanv0, but I will report an error when I run it:
ResourceExhaustedError: OOM when allocating tensor with shape[408576,32] [[Node: gradients/dense/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/gpu:0"](Reshape, gradients/dense/BiasAdd_grad/tuple/control_dependency)]] Caused by op 'gradients/dense/MatMul_grad/MatMul_1', defined at: File "/home/xuenzhu/anaconda3/lib/python3.6/sitepackages/spyder/utils/ipython/start_kernel.py", line 269, in <module> main() File "/home/xuenzhu/anaconda3/lib/python3.6/sitepackages/spyder/utils/ipython/start_kernel.py", line 265, in main kernel.start()
The Wrong sentence1:
self.r: self.r_buffer
The Wrong sentence2:
model.train()
This is the complete codeļ¼
#!/usr/bin/env python3 # * coding: utf8 *[enter image description here][1] import tensorflow as tf import numpy as np import gym import sys MODEL_SAVE_PATH="MNIST_model/" MODEL_NAME="mnist_model" sys.path.append('.') class Agent(object): def __init__(self, a_space, s_space, **options): self.session = tf.Session() self.a_space, self.s_space = a_space, s_space self.s_buffer, self.a_buffer, self.r_buffer = [], [], [] self.filter1=tf.Variable(tf.random_normal([3,3,1,32])) self.filter2=tf.Variable(tf.random_normal([3,3,32,64])) self._init_options(options) self._init_input() self._init_nn() self._init_op() def _init_input(self): self.s = tf.placeholder(tf.float32, [None, 88,80,1]) self.r = tf.placeholder(tf.float32, [None, ]) self.a = tf.placeholder(tf.int32, [None, ]) def _init_nn(self): # Kernel init. w_init = tf.random_normal_initializer(mean=0.0, stddev=0.3) hidden0 = tf.nn.conv2d(self.s,self.filter1,strides=[1, 1, 1, 1], padding='VALID' ) relu1 = tf.nn.relu(hidden0) hidden1 = tf.nn.conv2d(relu1,self.filter2,strides=[1, 1, 1, 1], padding='VALID' ) relu2 = tf.nn.relu(hidden1) pool1 = tf.nn.max_pool(relu2,ksize=[1,1,1,1],strides=[1,1,1,1],padding='SAME') hidden11=pool1.get_shape().as_list() #print(hidden11) reshaped=tf.reshape(pool1,[1,hidden11[1]*hidden11[2]*hidden11[3]]) # Dense 2. dense_2 = tf.layers.dense(reshaped, 32, tf.nn.relu, kernel_initializer=w_init) dense_3 = tf.layers.dense(dense_2, 200, tf.nn.relu, kernel_initializer=w_init) # Action logits. self.a_logits = tf.layers.dense(dense_3, self.a_space, kernel_initializer=w_init) # Action prob. self.a_prob = tf.nn.softmax(self.a_logits) def _init_op(self): # One hot action. action_one_hot = tf.one_hot(self.a, self.a_space) # Calculate cross entropy. cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=action_one_hot, logits=self.a_logits) self.loss_func = tf.reduce_mean(cross_entropy * self.r) self.train_op = tf.train.AdamOptimizer(self.learning_rate).minimize(self.loss_func) self.session.run(tf.global_variables_initializer()) def _init_options(self, options): try: self.learning_rate = options['learning_rate'] except KeyError: self.learning_rate = 0.001 try: self.gamma = options['gamma'] except KeyError: self.gamma = 0.95 def predict(self, state): action_prob = self.session.run(self.a_prob, feed_dict={self.s: state[np.newaxis, :]}) return np.random.choice(range(action_prob.shape[1]), p=action_prob.ravel()) def save_transition(self, state, action, reward): self.s_buffer.append(state) self.a_buffer.append(action) self.r_buffer.append(reward) def train(self): # Copy r_buffer r_buffer = self.r_buffer # Init r_tau r_tau = 0 # Calculate r_tau for index in reversed(range(0, len(r_buffer))): r_tau = r_tau * self.gamma + r_buffer[index] self.r_buffer[index] = r_tau # Minimize loss. _, loss = self.session.run([self.train_op, self.loss_func], feed_dict={ self.s: self.s_buffer, self.a: self.a_buffer, # Predict action. self.r: self.r_buffer }) # self.s_buffer, self.a_buffer, self.r_buffer = [], [], [] import matplotlib.pyplot as plt #%matplotlib inline env = gym.make('MsPacmanv0') env.seed(1) env = env.unwrapped model = Agent(env.action_space.n, env.observation_space.shape[0]) r_sum_list, r_episode_sum = [], None max_reward = 1.0 mspacman_color = np.array([210,164,74]).mean() for episode in range(10000): # Reset env. s, r_episode = env.reset(), 0 # Start episode. while True: # if episode > 80:()() # env.render() # Predict action. img=s[1:176:2,::2] img = img.mean(axis=2) mspacman_color = np.array([210,164,74]).mean() img[img==mspacman_color]=0 img = (img128)/1281 s=img.reshape(88,80,1) a = model.predict(s) # Iteration. s_n, r, done, _ = env.step(a) if done: r = 5 r_episode += r # Save transition. model.save_transition(s, a, r) s = s_n if done: if r_episode_sum is None: r_episode_sum = sum(model.r_buffer) else: r_episode_sum = r_episode_sum * 0.99 + sum(model.r_buffer) * 0.01 #r_episode_sum = r_episode_sum * 0.01 + sum(model.r_buffer) * 0.99 r_sum_list.append(r_episode_sum) break # Start train. model.train() r_episode1= r_episode if r_episode1>max_reward: max_reward = r_episode1 print('the max reward is:',r_episode1) if max_reward>2000.0: obs = env.reset() while True: env.render() a = model.predict(obs) # Iteration. s_n, r, done, _ = env.step(a) obs = s_n if done: break

Regression like quantification of the importance of variables in random forest
Is it possible to quantify the importance of variables in figuring out the probability of an observation falling into one class. Something similar to Logistic regression.
For example: If I have the following independent variables 1) Number of cats the person has 2) Number of dogs a person has 3) Number of chickens a person has
With my dependent variable being: Whether a person is a part of PETA or not
Is it possible to say something like "if the person adopts one more cat than his existing range of animals, his probability of being a part of PETA increases by 0.12"
I am currently using the following methodology to reach this particular scenario: 1) Build a random forest model using the training data 2) Predict the customer's probability to fall in one particular class(Peta vs non Peta) 3) Artificially increase the number of cats owned by each observation by 1 4) Predict the customer's new probability to fall in one of the two classes 5) The average change between (4)'s probability and (2)'s probability is the average increase in a person's probability if he has adopted a cat.
Does this make sense? Is there any flaw in the methodology that I haven't thought of? Is there a better way of doing the same ?

TypeError with Python : str and int
I received this error when trying to compile my code. I extracted data from xlsx file and created a dataframe ,replaced null values with 0, converted all the values to sting to be able to scatterplot and when i tried to show the results of my linear regression I received this error.
TypeError: unsupported operand type(s) for /: 'str' and 'int'
and this is the code I did so far
import pandas as pd import numpy as np import matplotlib.pyplot as plt def predict(x): return slope * x + intercept from scipy import stats xlsxfile = pd.ExcelFile("C:\\Users\\AchourAh\\Desktop\\PL14_IPC_03_09_2018_SP_Level.xlsx") data = xlsxfile.parse('Sheet1', index_col = None, header = None) data1 = data.fillna(0) #Replace null values of the whole dataset with 0 data1 = data1.astype(str) print(data1) X = data1.iloc[0:len(data1),1] print(X) Y = data1.iloc[0:len(data1),2] print(Y) axes = plt.axes() axes.grid() plt.scatter(X,Y) slope, intercept, r_value, p_value, std_err = stats.linregress(X, Y)
To notice that I am a beginner with this. The last line is causing the error This is the first columns COP COR and PAUS of the dataframe which I am trying to apply some Linear regression on:
0 PP SP000045856 COP COR SP000045856 PAUS 1 201723 0 2000 2 201724 12560 40060 3 201725 17760 15040 4 201726 5840 16960 5 201727 10600 4480 6 201728 0 14700 7 201729 4760 46820
... till line 27

Fast RCNN ROI Backpropagation
In the Fast RCNN paper, in the section 2.3  "backpropagation through ROI pooling layers", this formula is being used for calculating the partial derivative.
As far as my understanding, the formula calculates the derivative for the each activation input(the value present in the conv feature map), if that value was being selected as max value by the ROI pooling function.("the partial derivative dL/dyrj is accumulated if i is the argmax selected for yrj by max pooling").
So, for each subwindow being used for maxpool, there will be 49 derivatives being calculated as each subwindow outputs (7 x 7) feature map after ROI pooling.
Would someone please clarify how correct is this interpretation of the formula, and throw some more details on this.

LSTM Training Input Versus Live Evaluation Input  Dynamic RNN?
I am having trouble wrapping my head around RNNs for this problem.
The problem: live binary classification of video using image sequences. Meaning I am receiving a video one image at a time and need to predict either Class A or Class B for the most recent image received.
Current Solution
Training  I use a CNN as feature extractor on a full sequence of images. I then feed multiple images (lstmlen, cnnfeaturesize) into the LSTM.
Live Evaluation  I receive 1 frame at a time and run it through the CNN. I add these new features to a queue of length lstmlen, then I take all the features from the queue and feed into the LSTM.
What I don't understand
Why is it that I have to keep track of and feed all of the features into the LSTM at evaluation time? The point of an LSTM is to remember past inputs so it seems redundant for me to input all the previous images at every time step. What I would like to be able to do is simply calculate the features for the most recent images and then feed those new features into the LSTM while the LSTM remembers the last lstmlen number of frames.
Am I using the RNN incorrectly in this case? Should I be able to simply use the previous LSTM state as input into the other LSTM cells and provide feature input for only the newest image?
I'm thinking something like tensorflows DynamicRNN may be the solution to this problem.
Pretty confused about this. Thanks for the help!

what functions of OPENCV mainly used in image analysis or particularly in deep learning to improve accuracy?
I want to know about what functions or how we use OPENCV in analysis the image which is overall beneficial for deep learning models in gaining accuracy? and also particularly for image dataset?

Opencv Video Analysis Audio Files
I am using opencv for analysing a video. However, after I use the cv2.VideoCapture() function(analysing using YOLO Darknet) I am unable to retrieve the audio files. How do I attach the audio files so that it plays like a normal audio file.
import cvlib as cv from cvlib.object_detection import draw_bbox import cv2 # open webcam webcam = cv2.VideoCapture("143055.crf") if not webcam.isOpened(): print("Could not open webcam") exit() image_counter=0 # loop through frames while webcam.isOpened(): # read frame from webcam status, frame = webcam.read() if not status: print("Could not read frame") exit() # apply object detection bbox, label, conf = cv.detect_common_objects(frame) print(bbox, label, conf) # draw bounding box over detected objects out = draw_bbox(frame, bbox, label, conf) # display output cv2.imshow("Realtime object detection", frame) cv2.imwrite('/Users/dukeglacia/Downloads/test_images2/%d.jpg'%image_counter,frame) image_counter+=1 # press "Q" to stop if cv2.waitKey(1) & 0xFF == ord('q'): break # release resources webcam.release() cv2.destroyAllWindows()
As far as I know cv2 only extracts the frames and not the audio hence when I convert the frames back to video I obtain a noaudio video.

Is keras 1.2.2 compatible with tensorflow 1.10.0 or any other latest version of tensorflow
is it required for the keras to be of the latest version of tensorflow or will keras 1.2.2 work fine with tensorflow 1.10.0 as the backend

Matlab: make predictions with SVM for multiclass classification problems
I am trying to use a Support Vector Machine to classify my data in 3 classes. I used this Matlab function to train and crossvalidate the SVM:
Mdl = fitcecoc(XTrain, yTrain, 'Learners', 'svm', 'ObservationsIn', 'rows', ... 'ScoreTransform', 'invlogit','Crossval','on', 'Holdout', 0.2);
where XTrain contains all of my data and yTrain is a cell containing the names of each class to be assigned to the input data in XTrain. The function above returns to me:
Mdl > 1x1 ClassificationPartitionedECOC
My question is, what function do I have to use in order to make predictions using new data? In the case of binary classification, I build the SVM with 'fitcsvm' and then I predicted the labels with:
[label, score] = predict(Mdl, XTest);
However, if I feed the ClassificationPartitionedECOC to the 'predict' function, it gives me this error:
No valid system or dataset was specified.
I haven't been able to find a function that allows me to perform prediction starting from the model format that I have, ClassificationPartitionedECOC. Thanks for any help you may provide!

How to use classification Learner in my environment?
I am working on facial expression recognition. i made a dataset contain features & classes of 213 images.
 Step1: Each row of my dataset represents the features of 1 image. so for 213 images 213 rows
 Step2: the last column represents classes like; 1,2,3,4,5,6,7
 Q1: when i run classification learner and the cross validation is checked the accuracy is very low.
 Q2: when i run classification learner and the cross validation is not checked the accuracy is good for all classifiers.
I dont understand why? please help me? if someone has already experienced.

multiclass classification algorithm in R
I have a data with 289070 observations and 17 variables which most of them are categorical. Which machine learning algorithm can I use for multiclass classification problem because my input variable is multi class. I tried random forest but I have a variable which have 206 levels so R couldn't run it because it's more the 53 levels.