MLP performing worse than the SGD in a supervised classification
The learning dataset I'm using is a grayscale image that was
flatten to have each pixel representing an individual sample. The second image will be classified pixel by pixel after training the
Stochastic gradient descent (
Multilayer perceptron (
MLP) classifiers on the former one.
The problem I have is that the
SGD is performing way better than the
MLP, even if I keep the default parameters provided by
Scikit-learn in both cases. Below is the code for both (please notice that because the training dataset is in the order of millions of samples, I had to employ
partial_fit() to train the
MLP by chunks, while this was not necessary for the
def batcherator(data, target, chunksize): for i in range(0, len(data), chunksize): yield data[i:i+chunksize], target[i:i+chunksize] def classify(): if algorithm == 'sgd': classifier = SGDClassifier(verbose=True) classifier.fit(training.data, training.target) elif algorithm == 'mlp': classifier = MLPClassifier(verbose=True) gen = batcherator(training.data, training.target, 1000) for chunk_data, chunk_target in gen: classifier.partial_fit(chunk_data, chunk_target, classes=np.array([0, 1]))
My question is which parameters should I adjust in the
MLP classifier to make its results similar to those obtained with the
I've tried to increase the number of neurons in the hidden layer using
hidden_layer_sizes but I didn't see any improvement. No improvement either if I change the activation function of the hidden layer from the default
logistic using the
See also questions close to this topic
How to install specific function from package in R?
I have installed 'caret' package using
install.packages('caret', dependencies = c("Depends", "Suggests"))
But I didn't get createDataPartition and dummyVars functions part of 'caret' package.
> dummyVars(formula, data, sep = ".", levelsOnly = FALSE, fullRank = FALSE, ...) >createDataPartition(y, times = 1, p = 0.5, list = TRUE, groups = min(5, length(y)))
even while loading package
library('caret')it is giving :
Error: package or namespace load failed for ‘caret’ in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]): there is no package called ‘ddalpha’ In addition: Warning message: package ‘caret’ was built under R version 3.4.3
And I installed many packages that I got in error message.
So I want to know, Is there any way to install specific function from package?
extracting patterns from a system using machine learning
The following is the problem statement: I have a system in which I observe and extract various events. A group of events are highly related in time. The groups themselves are not in any way related. For example, let us assume that the system has two blocks - A and B. An event from A (say A.e1) occurs within a time window t1; in response to this, event from B (say B.e2) occurs within a time window t2; in response to this event from A (say A.e3) occurs within a time window t3. So, this is one group of events or one pattern. If a system has many such block (like A and B), many such event groups would happen. The observer in the system generates log of these events when system is operational. A human can look at these events in the log and identify the patterns. What would be the best ML approach for a problem of this type? PS: I am not expecting a solution but more of direction for such problem class.
How to launch oython projects?
I am studing python for a week and machine learning for about 2 days so sorry for asking stupid question. I wanted to learn machine learning via one project on github but I am totally lost how to launch python projects and what is their structure overall...
For example https://github.com/MaxTitkov/Keras_InceptionV3_Binary_classification lots of python files and I can not understand which one to launch first so it will pull up the rest.. None of them has reference to the others files inside the code. So I guess I need to launch them all separately via python + name of the file or how does it work with python projects?
I found this text in sklearn.cross_validation module, train_test_split section:
import numpy as np x, y = np.arange(10).reshape(5,2), range(10) print(x)
why should i write range here?
if i don't write range the output error says:
ValueError Traceback (most recent call last) <ipython-input-20-5b1b95db99cc> in <module>() 1 import numpy as np ----> 2 x, y = np.arange(10).reshape(5,2) 3 print(x) ValueError: too many values to unpack (expected 2)
python - determine the number of languages in a given sentence
I need to determine the number of languages in a sentence.
For example I have the sentence "Hello друг" - here I have 2 lang: EN, RU
Function will return bool value "if all the words in the same language"
I wrote a function, but it's slow. My problem is that I have a lot of such sentences and it works very slowly
No verbose for Nu_SVR in scikit-learn
I'm using SVM in scikit learn, and need to have verbose to see how my model is doing while training. But i can't get the verbose in SVM and i keep getting the following output :*
begin training .... [LibSVM]
this is the code i wrote :
print("begin training ....") svm = NuSVR(kernel="rbf",C=20, nu=0.9,verbose=True) svm.fit(trX,trY) print(".... training ended")
so why i'm getting [LibSVM] instead of verbose? and what should i do to get a verbose ?
AWS Sagemaker Neural Topic Model
What is underlying algorithm for Sagemaker's Neural Topic Model? I have hard time googling for details, and the documentation doesn't mention any paper.
Googling for 'neural topic model' doesn't exactly answer my question, since a couple of methods seems to be called that.
Predictions in the mxnet package
I'm trying to train a neural network and convolutional neural network in R with the package mxnet for a 2-class classification problem. I'm using the fucntion "mx.mlp" for the neural network and the function "mx.model.FeedForward.create" for the convolutional neural network (CNN).
My problem is the predictions. I get the exact same probabilities for the 2 classes from both of the 2 functions in the test set which is very strange. This means that the models can only predict one class. I have tried to solve this problem for 3 weeks now without success. Any help would be much appreciated.
here is the links to the:
How to enforce rules like move legality in chess at the output of a neural network?
How do I apply rules, like chess rules, to a neural network, so the network doesn't predict/train invalid moves?
Can "Sequential Feature Selector" be used for one-class feature selection?
Can the methods presented here be used for feature selection for one-class classification?
Feature selection for one-class classification in Python
I am seeking feature selection methods for one-class classification using Python. Would feature importance in random forest classifier in scikit-learn also work for one-class problem?
I haven't managed to find an example of that...
how to find which rules in decision tree are causing misclassifications
I built an binary decision tree classifier . From the confusion matrix m i found class 0 is misclassified 495 times and class 1 is misclassified 134 times.I want to find which rules in the decision trees are actually causing the records to misclassify.
In short which record failed at the which tree node
Is there a machine learning method which can be used to find the rules in the decision tree which are causing them to misclassify
[[14226 495] [ 134 3271]]
Fitting the decision tree and plotting it
cv = CountVectorizer( max_features = 200,analyzer='word',ngram_range=(1, 3)) cv_addr = cv.fit_transform(data.pop('Clean_addr')) for i, col in enumerate(cv.get_feature_names()): data[col] = pd.SparseSeries(cv_addr[:, i].toarray().ravel(), fill_value=0) train = data.drop(['Resi], axis=1) Y = data['Resi'] X_train, X_test, y_train, y_test = train_test_split(train, Y, test_size=0.3,random_state =8) rus = RandomUnderSampler(random_state=42) X_train_res, y_train_res = rus.fit_sample(X_train, y_train) dt=DecisionTreeClassifier(class_weight="balanced", min_samples_leaf=30) fit_decision=dt.fit(X_train_res,y_train_res) from sklearn.externals.six import StringIO from IPython.display import Image from sklearn.tree import export_graphviz import pydotplus dot_data = StringIO() export_graphviz(fit_decision, out_file=dot_data, filled=True, rounded=True, special_characters=True,feature_names=train.columns) graph = pydotplus.graph_from_dot_data(dot_data.getvalue()) Image(graph.create_png())from sklearn.externals.six import StringIO from IPython.display import Image from sklearn.tree import export_graphviz import pydotplus dot_data = StringIO() export_graphviz(fit_decision, out_file=dot_data, filled=True, rounded=True, special_characters=True,feature_names=train.columns) graph = pydotplus.graph_from_dot_data(dot_data.getvalue()) Image(graph.create_png())
Any help is appreciated.
Resi is the target column . Using the other data columns i am trying to predict and I have countvectorized the Clean_addr column.
How does Caffe updates gradients of a blob with more than one output branch？
Caffe supports multiple losses. Then for the backpropagation stage, some blobs may have multiple gradients coming from different losses. How does Caffe do with the gradients of this blob?
As far as I know, this may not be a concern when designing networks. But this question really confuse me when I try to write a new layer. Thanks for any idea!
back propagation on lambda Keras
In the neural network, there is a lambda layer, that computes the normalized Euclidean distances between the outputkvector (vector output of the neural network) and classkvector (matrix of vectors representing each class).
Where outputkvector is the output of the whole neural network, which I’m parameterizing with V, representing all the coefficients.
How to take the gradient with respect to both V and classkvector? By which I mean all the parameters in V and all the elements of the classkvector matrix (each treated as variables)
Logistic Regression Gradient Descent
I am trying to do as the title says but I am not able to really understand how to do it.
If you could help me understand what's wrong would be very helpful. I have to do Logistic regression using batch gradient descent.
import matplotlib.pyplot as plt import numpy as np %matplotlib inline X = np.asarray([ [0.50],[0.75],[1.00],[1.25],[1.50],[1.75],[1.75], [2.00],[2.25],[2.50],[2.75],[3.00],[3.25],[3.50], [4.00],[4.25],[4.50],[4.75],[5.00],[5.50]]) y = np.asarray([0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,1,1,1,1,1]) m = len(X) def sigmoid(a): return 1.0 / (1 + np.exp(-a)) def gradient_Descent(theta, alpha, X , y): for i in range(0,m): cost = ((-y) * np.log(sigmoid(X[i]))) - ((1 - y) * np.log(1 - sigmoid(X[i]))) grad = theta - alpha * (1.0/m) * (np.dot(cost,X[i])) theta = theta - alpha * grad return gradient_Descent(0.1,0.005,X,y)
The way I have to do it is like this but I can't seem to understand how to make it work.