ROC or CAP CURVE for a multiclass classification in python
I am unable to plot ROC curve for a multiclass problem.
See also questions close to this topic
pandas create new column based on grouping
I have a pandas dataframe which looks like the following.
df = index| date | amount| type| 0 | 2015 | 1000 | A | 1 | 2015 | 100 | B | 2 | 2016 | 3500 | A | 3 | 2017 | 150 | C |
I want to produce a dataframe with a new column based on types. The result would look like the following.
df = index| date | A | B | C 0 | 2015 | 1000| 100 |0 1 | 2016 | 3500| 0 |0 2 | 2017 | 0 | 0 |150
Assign unique value out of multiple columns (allowed values)
So I would assign either vertex 4 or 2 to edge 0, vertex 1 or 2 to edge 1, etc. But, every edge should have an unique vertex assigned to them!
So if I take edge0 = vertex4, edge1 = vertex1, edge2 = vertex3, ... then edge3 can only choose between vertex 1 or 4, which is already taken by one of the other edges.
In this case edge0 = vertex2, edge1 = vertex1, edge2 = vertex3 and edge3 = vertex4, would work!
In the end there will still be combinations that will leave an "edge" without a unique vertex, but then this edge should just get a NaN value or so.
Thanks in advance for the help!
Something wrong with PyQt5 menuBar
There were something wrong when I was using PyQt5 to build a GUI window with a menu bar.
Here is my code:
import sys from PyQt5.QtWidgets import QMainWindow, QAction, QApplication class Example(QMainWindow): def __init__(self): super().__init__() self.initUI() def initUI(self): bar = self.menuBar() example1 = QAction('Exit', self) example1.setShortcut('Ctrl+E') example1.triggered.connect(self.close) example2 = QAction('xit', self) example2.setShortcut('Ctrl+A') example2.triggered.connect(self.close) example3 = QAction('Quit', self) example3.setShortcut('Ctrl+Q') example3.triggered.connect(self.close) fileMenu = bar.addMenu('File') fileMenu.addAction('NNN') fileMenu.addAction(example1) fileMenu.addAction(example2) fileMenu.addAction(example3) self.setGeometry(300, 300, 300, 200) self.setWindowTitle('Menu Example') self.show() if __name__ == '__main__': app = QApplication(sys.argv) ex = Example() sys.exit(app.exec_())
When I run this, the menu bar looks like this:
As the picture shows, 'Exit' and 'Quit' disappeared, but the shortcut worked.
My Env: Python 3.6.5, PyQt 5.11.1， MAC_OS 10.13.5
Error in prediction(preds, testdata[, 2]) : Format of predictions is invalid
I am getting following error in R
Error in prediction(preds, testdata[, 2]) : Format of predictions is invalid.
I have written following code in order to draw AUC
# SVM library(e1071) classifier = svm(formula = class~.,data = traindata, type = 'C-classification', kernel = 'linear') y_pred = predict(classifier,newdata=testdata[-2]) cm = table(testdata[,2],y_pred) cm conf_m = confusionMatrix(table(testdata[,2],y_pred)) prediction(preds,testdata[,2]) # Getting error in this line performance(prediction(preds,testdata[,2]),"tpr","fpr") plot(performance(ROCR::prediction(preds,testdata[,2]),"tpr","fpr")) abline(a=0,b=1) auc<-performance(ROCR::prediction(preds,testdata[,2]),"auc") auc
For solving this error I have tried to check class of
class(testdata[,2]). It is not vector, matrix, list, data frame. So I tried to convert it into numeric.
But then it gives another error
Error in prediction(as.numeric(preds), as.numeric(testdata[, 2])) : Number of predictions in each run must be equal to the number of labels for each run.
Using ROC value as a threshold in confusion matrix
Is it possible to use AUC value as a threshold to build the confusion matrix?
Error in performance(pred, "tpr", "fpr")
I am trying to plot an ROC curve using the ROCR package in R but am stuck with the following error:
Error in performance(pred, "tpr", "fpr") : Assertion on 'pred' failed: Must have class 'Prediction', but has class 'prediction'
This is the code I am using to get to the performance segment call:
#convert actual and predicted labels to numeric predicted<-as.numeric(as.character(test$Class)) actual<-as.numeric(as.character(test$overall_satisfaction)) #generate confusion matrix and label positive class library(caret) confusionMatrix(predicted,actual,positive="1")
The confusion matrix output looks just fine. However, in the next segment, performance function in ROCR throws an error and as a consequence, the ROC curve is not plotted.
#ROC curve library(ROCR) pred<-prediction(predicted, actual) perf<-performance(pred,"tpr","fpr") plot(perf,col="red", main="mlr_parameters_ROC") abline(0,1, lty = 8, col = "grey")
I am unable to figure out what is wrong with the above code. Can somebody please help?
glmnet multinomial logistic regression prediction result
I'm building a penalized multinomial logistic regression, but I'm having trouble coming up with a easy way to get the prediction accuracy. Here's my code:
fit.ridge.cv <- cv.glmnet(train[,-1], train[,1], type.measure="mse", alpha=0, family="multinomial") fit.ridge.best <- glmnet(train[,-1], train[,1], family = "multinomial", alpha = 0, lambda = fit.ridge.cv$lambda.min) fit.ridge.pred <- predict(fit.ridge.best, test[,-1], type = "response")
The first column of my test data is the response variable and it has 4 categories. And if I look at the result(fit.ridge.pred) it looks like this:
1,2,3,4 0.8743061353, 0.0122328811, 0.004798154, 0.1086628297
From what I understand these are the class probabilities. I want to know if there's a easy way to compute the model accuracy on the test data. Now I'm taking the max for each row and comparing with the original label. Thanks
Multi-class Clasification (multiclassification): Micro-Average Accuracy, Precision, Recall and F Score All Equal
I am testing the performance of two multinomial text classifiers. The original corpus used to do this contains 8 classes. This is split into training and test data. The training data has been oversampled using SMOTE to correct for the class imbalance but not for testing.
I have produced the confusion matrices for different text classifiers but did not calculate the performance metrics in Python (this was an oversight). Since the analysis I have performed to date has been extremely time consuming and I do not want to repeat it again, I am calculating the micro-average of several metrics manually. This includes Precision, Recall and F score (the micro-average accuracy was computed using the sklearn.metric libary in Python).
Below is a screenshot of the manual calculations performed:
FN = sum of row minus TP
FP = sum of columns minus TP
The calculations for the micro-average values are:
Recall: TP/(TP+FN) = 2388/(2388+117) = 0.9532
Precision: TP/(TP+FP) = 2388/(2388+117) = 0.532
F1: harmonic mean of the Recall and Precision = 0.9532
Accuracy: TP/(P+N) = 2388/2505 = 0.9532
Am I computing something incorrectly or is this a known feature of micro-average metrics (specifically Recall, Precision, F Score and Accuracy), i.e. they are all equal? As you can see from the screenshot, I also compute macro-average scores, which are different but since there is a class imbalance, I am using micro-average metrics.
I can see from the below link on for sklearn.metric that the micro-average Precision, Recall and F-score are equal to each other:
Note that for “micro”-averaging in a multiclass setting with all labels included will produce equal precision, recall and F
With that said, should it also equal to the micro-average Accuracy? I would be grateful for any literature that defends this.
Object detection with multi label classification
I am doing a project on Damage car detection. I want to detect various car areas(panels such as bumper,door,windshield,etc) which are damaged in an image and also determine the level of damage severity(minor, major). Can anyone tell me how can I train my own dataset with multiple labels for object detection?
For example, if the door is damaged and damage is minor the output of classification should have probabilities of both door and damage(minor) with boxe around door as well as box around damaged area(like scratches,dent,etc) and also classify it as door and minor. Same goes for other parts of the vehicle such as bumper, headlights, etc. I've got data for both.I just need an approach. 1) Can it be done with YOLO or other object detection method. 2) Is it multi-class multi-label problem
Any other approach/suggestions are welcomed. Any link to papers, blogs or repo will be helpful.
- npm cap is warning for windows ( network traffic listening )
CAP theorem in case master is down under RDBMS?
- Consider a 3 node(N1,N2,N3) system
- N1 is master(used for writes) and N2,N3(used for reads) are slaves
- Write request comes to update
R1, N1 writes to disk and crashed without replication
- Will system continue to work and Oracle select the another master automatically ?
- Write request comes to update
- N1 is back again
Which value will be kept -
I am using oracle as RDBMS.
When I develop javacard on Eclipse it's say " Internal error occured, look for the cap conversion error "
When i develop javacard on Eclipse,it's say "Internal error occured, look for the cap conversion error !". Platform is Windows.