How to apply the learned model using boosting technique

See also questions close to this topic

• Hierarquical clustering with big vector

  I'm working in MATLAB and I have a vector of 4 million values. I've tried to use the linkage function but I get this error:

  Error using linkage (line 240) Requested 1x12072863584470 (40017.6GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

  I've found that some people avoid this error using the kmeans functions however I would like to know if there is a way to avoid this error and still use the linkage function.
• I have two tall arrays and i want to find the intersection of them matlab

  In common tables the intersection function would have worked great, but for tall tables this function has not been implemented yet. So is there an other way to find the common elements of these two???

  I use MatLab2017a.
• Unable to run Java code on MATLAB Runtime in Unix environment

  After exporting a matlab function to java package. I have tried to run the following code unix environment which contains MATLAB runtime.

  import com.mathworks.toolbox.javabuilder.MWArray;
  import com.mathworks.toolbox.javabuilder.MWCharArray;
  import calibrationAPINew.Calibration;
  
  public class CHIndexCalculation {
          public static void main(String[] args) {
  
                MWCharArray number1 = null;
                Object[] result = null;
                Calibration calibration = null;
                String data = "{\r\n" +
                          "\"tsId\" : \"121213\",\r\n" +
                          "\"datapoints\" : [0.1,0.2,0.3,0.4]\r\n" +
                          "}";
  
                try
                {
                   System.out.println("Input data: " + data);
                   number1 = new MWCharArray(data); // Array of Input to be sent to MATLAB Runtime
                   System.out.println("MWCharArray: " + number1);
                   calibration = new Calibration();
                   result = calibration.calibrationAPI(1,number1);
  
                   System.out.println("Output : " +  result[0]);
                }
                catch (Exception e)
                {
                  System.out.println("Exception Occurred : " + e);
                   e.printStackTrace();
                }
                finally
                {
                   //MWArray.disposeArray(number1);
                   //MWArray.disposeArray(result);
                   //calibration.dispose();
                }
  
          }
  }
  

  I have installed the matlabruntime on unix box and set the environment variables using export statement

  export LD_LIBRARY_PATH=matlabrt/v92/runtime/glnxa64:matlabrt/v92/bin/glnxa64:matlabrt/v92/sys/os/glnxa64:matlabrt/v92/sys/opengl/lib/glnxa64
  

  Now when I try to run the command ( calibrationAPINew.jar is the jar file exported from MATLAB)

  java -cp javabuilder.jar:calibrationAPINew.jar:. CHIndexCalculation
  

  I get the following exception

  com.mathworks.toolbox.javabuilder.MWException: Undefined function 'pdist2mex' for input arguments of type 'double'.
          at com.mathworks.toolbox.javabuilder.internal.MWMCR.mclFeval(Native Method)
          at com.mathworks.toolbox.javabuilder.internal.MWMCR.access$600(MWMCR.java:31)
          at com.mathworks.toolbox.javabuilder.internal.MWMCR$6.mclFeval(MWMCR.java:882)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at com.mathworks.toolbox.javabuilder.internal.MWMCR$5.invoke(MWMCR.java:769)
          at com.sun.proxy.$Proxy0.mclFeval(Unknown Source)
          at com.mathworks.toolbox.javabuilder.internal.MWMCR.invoke(MWMCR.java:443)
          at calibrationAPINew.Calibration.calibrationAPI(Calibration.java:223)
          at CHIndexCalculation.main(CHIndexCalculation.java:24)
  

  I know this has something to do with MATLAB runtime but I am unable to figure out what ? The same set up works fine on windows environment.
• RandomForestClassifier instance not fitted yet. Call 'fit' with appropriate arguments before using this method

  I am trying to train a decision tree model, save it, and then reload it when I need it later. However, I keep getting the following error:

  This DecisionTreeClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this method.

  Here is my code:

  X_train, X_test, y_train, y_test = train_test_split(data, label, test_size=0.20, random_state=4)
  
  names = ["Decision Tree", "Random Forest", "Neural Net"]
  
  classifiers = [
      DecisionTreeClassifier(),
      RandomForestClassifier(),
      MLPClassifier()
      ]
  
  score = 0
  for name, clf in zip(names, classifiers):
      if name == "Decision Tree":
          clf = DecisionTreeClassifier(random_state=0)
          grid_search = GridSearchCV(clf, param_grid=param_grid_DT)
          grid_search.fit(X_train, y_train_TF)
          if grid_search.best_score_ > score:
              score = grid_search.best_score_
              best_clf = clf
      elif name == "Random Forest":
          clf = RandomForestClassifier(random_state=0)
          grid_search = GridSearchCV(clf, param_grid_RF)
          grid_search.fit(X_train, y_train_TF)
          if grid_search.best_score_ > score:
              score = grid_search.best_score_
              best_clf = clf
  
      elif name == "Neural Net":
          clf = MLPClassifier()
          clf.fit(X_train, y_train_TF)
          y_pred = clf.predict(X_test)
          current_score = accuracy_score(y_test_TF, y_pred)
          if current_score > score:
              score = current_score
              best_clf = clf
  
  
  pkl_filename = "pickle_model.pkl"  
  with open(pkl_filename, 'wb') as file:  
      pickle.dump(best_clf, file)
  
  from sklearn.externals import joblib
  # Save to file in the current working directory
  joblib_file = "joblib_model.pkl"  
  joblib.dump(best_clf, joblib_file)
  
  print("best classifier: ", best_clf, " Accuracy= ", score)
  

  Here is how I load the model and test it:

  #First method
  with open(pkl_filename, 'rb') as h:
      loaded_model = pickle.load(h) 
  #Second method 
  joblib_model = joblib.load(joblib_file)
  

  As you can see, I have tried two ways of saving it but none has worked.

  Here is how I tested:

  print(loaded_model.predict(test)) 
  print(joblib_model.predict(test)) 
  

  You can clearly see that the models are actually fitted and if I try with any other models such as SVM, or Logistic regression the method works just fine.
• Handling outlier for e-commerce data

  I have read about deleting the data points that is outlier, I also know about winsorizing the data also to change the upper and lower inner fence.

  Is there any better solution for handling the outlier in e-commerce dataset? Thank you :)
• ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() when converting a map object to list

  I've title_result which is a list of 1000 movie titles.
  i've a function whivh return similar movies based on a model. The function looks like this:

  def get_recommendations(title):
      idx = indices[title]
      sim_scores = list(enumerate(cosine_sim[idx]))
      sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)
      sim_scores = sim_scores[1:31]
      movie_indices = [i[0] for i in sim_scores]    
      return title, str(titles.iloc[movie_indices].reset_index(drop=True).tolist())    
  

  The output of function will be as folllows when 'Twilight Saga: Breaking Dawn - 1' is passed as input parameter.
  ('Twilight Saga: Breaking Dawn - 1', "['Twilight Saga: Breaking Dawn - 2', 'Kedi Billa Killadi Ranga', 'Anjali', 'Half Girlfriend']")

  I've used a map function to get result for rest of the titles.

  result = list(map(get_recommendations,title_result))
  result_df = pd.DataFrame(result,columns=['title','similar_movies'])
  

  When I pass list range title_result[:20], I will get the output perfectly, but when when I pass whole list or list range from [25:30], I'm getting

    `ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()`.
  

  I think there is some exception with certain values when converting map object to list, which I'm unable to solve.
• Can change network architecture during training in tensorflow?

  I want to change channel dimensions of convolution layer during training, but in testing, always keeping same dimensions. I tried to implement this in tensorflow, and i failed. If the output dimension changes during training, an error occurs because the tensor flow does not recognize the changed graph. Does not the tensor flow implement these features? If not, is there any other way to implement this feature?
• Domain adaptation - Adversarial Dropout Regularization - Keras implementation of custom loss

  I am trying to implement in Keras domain adaptation method Adversarial Dropout Regularization from the paper https://arxiv.org/abs/1711.01575

  Critic is classifier with dropouts and it is updated based on losses from more then one batch. Idea is to get two outputs for one target batch (dropout is included in test phase ), and measure KL divergence between outputs (they will not be same because of dropouts). Additionally to this value, critic loss is also depended on source batch loss. The final loss value is subtraction of source batch loss and discrepancy between outputs of target batch. (Eq 3 and 4, page 4),

  I don't know how to implement this loss, due it depends on outputs of three batches (two target and one source).

  I tried to make one batch consisted of one source batch and two target batches (same target batch), and then in custom loss to separate y_pred and calculate measures. Here is my custom loss :

  def custom_loss(self,y_true,y_pred):
      pred_source = y_pred[:self.batchSize,:]
      pred_tgt_1 = y_pred[self.batchSize+1:2*self.batchSize,:]
      pred_tgt_2 = y_pred[2*self.batchSize+1:,:]
      discp = discrepancy(pred_tgt_1,pred_tgt_2)
      loss = keras.losses.categorical_crossentropy(y_true[:self.batchSize],pred_source)
  
  return  loss - discp
  

  I got error Incompatible shapes: [64] vs. [192] (batch size is 64 ).

  Dropout layers are added like this :

  cl = layers.Dropout(0.5)(cl,training=True)
  

  Can someone help me with this?
• How to get train accuracy in SSD for caffe version?

  I am testing SSD caffe version for object detection. This caffe version doesn't log into the file for accuracy in training. Just log detection evaluation in training. So I am trying to put similar layers in train.prototxt file as follow. But it produces a lot in each iterations, how can I have only one for each iteration.

  add in layers to train.prototxt file

  layer {
    name: "mbox_conf_reshape"
    type: "Reshape"
    bottom: "mbox_conf"
    top: "mbox_conf_reshape"
    reshape_param {
      shape {
        dim: 0
        dim: -1
        dim: 2
      }
    }
    propagate_down: false
  }
  layer {
    name: "mbox_conf_softmax"
    type: "Softmax"
    bottom: "mbox_conf_reshape"
    top: "mbox_conf_softmax"
    softmax_param {
      axis: 2
    }
    propagate_down: false
  }
  layer {
    name: "mbox_conf_flatten"
    type: "Flatten"
    bottom: "mbox_conf_softmax"
    top: "mbox_conf_flatten"
    flatten_param {
      axis: 1
    }
    propagate_down: false
  }
  layer {
    name: "detection_out"
    type: "DetectionOutput"
    bottom: "mbox_loc"
    bottom: "mbox_conf_flatten"
    bottom: "mbox_priorbox"
    top: "detection_out"
    include {
      phase: TRAIN
    }
    propagate_down: false
    propagate_down: false
    propagate_down: false
    detection_output_param {
      num_classes: 2
      share_location: true
      background_label_id: 1
      nms_param {
        nms_threshold: 0.449999988079
        top_k: 400
      }    
      code_type: CENTER_SIZE
      keep_top_k: 200
      confidence_threshold: 0.00999999977648
    }
  }
  layer {
    name: "detection_eval"
    type: "DetectionEvaluate"
    bottom: "detection_out"
    bottom: "label"
    top: "detection_eval"
    include {
      phase: TRAIN
    }
    propagate_down: false
    propagate_down: false
    detection_evaluate_param {
      num_classes: 2
      background_label_id: 1
      overlap_threshold: 0.3
      evaluate_difficult_gt: false
      name_size_file: "/home/coie/data/NumberPlate/test_name_size.txt"
      resize_param {
        prob: 1.0
        resize_mode: WARP
        height: 480
        width: 480
        interp_mode: LINEAR
        height_scale: 480
        width_scale: 480
      }
    }
  }
  
• Split data set and split ratio to get constant accuracy in Python

  def splitDataset(dataset, splitRatio):
    trainSize = int(len(dataset) * splitRatio)
    trainSet = []
    copy = list(dataset)
    while len(trainSet) < trainSize:
      index = random.randrange(len(copy))
      trainSet.append(copy.pop(index))
    return [trainSet, copy]
  

  Using this function I get different accuracy every time I run the classifier. It was caused by random function. So how can I split my data to get the unchanging accuracy? this line is the main problem :

  index = random.randrange(len(copy))
  
• Calculate Local Binary Pattern with radius

  I'm confused with a local binary pattern that uses radius. I have read the journal by Ojala et al, where they use this equation to calculate the center pixel :Eq.1

  

  
  Where :Eq.2

  

  
  with :
  R = Radius
  P = amount of neighborhood pixel
  gp = neighborhood pixel
  gc = center pixel

  for example R=2;P=16(from 0-15)
  if s(g15-gc)=1 and the other is 0
  
  so the LBP value is:
  LBP=1*2^15=32768

  so, how this big value turn into pixel value?
• sklearn adapt pipeline for multiple features

  This code works when x is a vector (ie single feature). How do I adapt this when x is an array (2 or more features). I want tfidf to vectorize each column and use both features in the classification model.

  The code runs as written but seems to ignore the extra features

  def grid_search(train_x, train_y):
  
      from sklearn.model_selection import GridSearchCV
      from sklearn.pipeline import Pipeline
      from sklearn.feature_extraction.text import TfidfVectorizer
      from sklearn.multiclass import OneVsRestClassifier
      from sklearn.naive_bayes import MultinomialNB
  
  
      parms={ 'tfidf__max_df': (0.25, 0.5, 0.75),
              'tfidf__ngram_range': [(1, 1), (1, 2), (1, 3)],
              'clf__estimator__alpha': (1e-2, 1e-3)
              } 
  
      pipeline = Pipeline([
          ('tfidf', TfidfVectorizer(stop_words=stop_words)),
          ('clf', OneVsRestClassifier(MultinomialNB( fit_prior=True, class_prior=None)))
          ])
      gs1 = GridSearchCV(pipeline, parms, cv=2, n_jobs=1, verbose=0)
      gs1.fit(train_x, train_y)
      return gs1.best_estimator_
  
  
  classifier = grid_search(train_x, y_train)
  
• How do I implement particle swarm optimisation on my adaboostclassifier to optimise the results

  from sklearn.pipeline import Pipeline
  pipeline = Pipeline([('bow', CountVectorizer(input = 'opinion',analyzer ='word',)),
                  ('tfidf', TfidfTransformer()),
                  ('classifier', AdaBoostClassifier(n_estimators = 300))])
  pipeline.fit(opinion_train,sentiment_train)
  predictions = pipeline.predict(opinion_test)
  from sklearn.metrics import classification_report
  

  NOTE: The code above is just the end part of code. Before this I read the data performed data cleaning and stuff. After this I want to perform pso (particle swarm optimisation) on adaboost. Can anyone of you tell me how to perform it. My dataset consisted of sentiment and tweets.
• Reuse stump in Adaboost

  The Adaboost algorithm works as below.

  Initialize weights D[i] = 1/m for all {D1, ..., Dm} FOR t in T: CHOOSE stump that minimizes error[t], sum D[i] of misclassified samples COMPUTE error[t] and alpha[t] UPDATE weights D[i] for all {D1, ..., Dm}

  After updating the weights D, it is possible that a stump still remains with the lowest error[t]. In respect to this situation, my question is: May I use the same stump while it keeps giving the lowest error, or should I ignore the stump and get a different one?
• Unable to understand testing on validation set after Combining pattern classifiers in ensemble learning

  I am trying to understand how to apply the learned models after training several different classifiers. In the end, we will have only one model obtained using the best combination method. The implementation that I am trying to follow is https://www.mathworks.com/matlabcentral/fileexchange/63162-adaboost?focused=7722161&tab=function

  CONFUSION 1) After the combination of the classifiers using the Weighted average method and the parameter alpha

  % final vote

  H(:,1)=predict(gda_in, Xtrain);
  H(:,2)=predict(knn_in, Xtrain);
  H(:,3)=predict(nb_in, Xtrain);
  H(:,4)=predict(linear_in, Xtrain);
  H(:,5)=predict(svm_in, Xtrain);
  ada_train(:,1)=sign(H*alpha');
  

  Is the learned model ada_train? If so, then why this learned model is not used in the prediction of new labels in the following step and thereby why do we need to compute ada_train?

  % for test set
  Htest(:,1)=predict(gda_in, Xtest);
  Htest(:,2)=predict(knn_in, Xtest);
  Htest(:,3)=predict(nb_in, Xtest);
  Htest(:,4)=predict(linear_in, Xtest);
  Htest(:,5)=predict(svm_in, Xtest);
  
  ada_test(:,1)=sign(Htest*alpha');
  

  CONFUSION 2) For the validation stage, where I have a new data set Xvalidate that has not been used in the training, how do I use the learned model to predict on this unseen data? In my understanding, I should use the second code snippet given above. But I am not sure because again I will not be using the ada_train variable.