Related questions

How to apply the learned model using boosting technique

2018-06-22 21:19 Srishti M imported from Stackoverflow

I have no clue on how to apply the learned model using adaptive boosting method since there is not model as such that is returned similar to the one returned by say the SVM classifier's fitcsvm() function. I am using the implementation given https://www.mathworks.com/matlabcentral/fileexchange/63162-adaboost?focused=7722161&tab=function

This method uses 5 base classifiers and so the number of learning cycles =5. Variable ada_test contains the labels from each independent run, itr of the boosting module.

QUESTION: For the boosting method once the training is done which includes the training set and prediction on the validation set, which variable should be used to predict on the unseen data set for testing? In the following code,

for itr=1:maxItr
    [~,ada_test(:,itr)]= adaboost(X,Y, Xtest);
    fm_=[fm_; confusion_mat(Ytest, ada_test(:,itr))];
end

the variable ada_test(:,1)=sign(Htest*alpha'); are the predicted labels. In the function, adaboost() there is the line ada_train(:,1)=sign(H*alpha'); Why ada_train variable is never used and how to test the learned adaboost model on unseen data? I cannot understand the procedure to predict on a new unseen test data because I cannot follow which variable or what is the learned model. Please help.

See also questions close to this topic

  • How to get data from python datatype returned in MATLAB?

    I have a python script like so:

    import numpy as np

def my_function(x):
    return np.array([x])

    And I have a MATLAB script to call it:

    clear all;
clc;
if count(py.sys.path,'') == 0
    insert(py.sys.path,int32(0),'');
end

myfunction_results = py.python_matlab_test.my_function(8);
display(myfunction_results);

    And it displays: 

    myfunction_results = 

  Python ndarray with properties:

           T: [1×1 py.numpy.ndarray]
        base: [1×1 py.NoneType]
      ctypes: [1×1 py.numpy.core._internal._ctypes]
        data: [1×8 py.buffer]
       dtype: [1×1 py.numpy.dtype]
       flags: [1×1 py.numpy.flagsobj]
        flat: [1×1 py.numpy.flatiter]
        imag: [1×1 py.numpy.ndarray]
    itemsize: 8
      nbytes: 8
        ndim: 1
        real: [1×1 py.numpy.ndarray]
       shape: [1×1 py.tuple]
        size: 1
     strides: [1×1 py.tuple]

    [8.]

    But I do not know how to actaully get the data out of this object. The type is py.numpy.ndarray, but I want to obviously use it in MATLAB as an array or matrix, or integer or something. HOw do I convert it to one of those types?

    I've been looking at these: https://www.mathworks.com/help/matlab/examples/call-python-from-matlab.html https://www.mathworks.com/matlabcentral/answers/216498-passing-numpy-ndarray-from-python-to-matlab https://www.mathworks.com/help/matlab/matlab_external/use-matlab-handle-objects-in-python.html

    Some of the answers suggest writing to a .mat file. I DO NOT want to write to a file. This needs to be able to run in real time and writing to a file will make it very slow for obvious reasons.

    Seems like there is an answer here: "Converting" Numpy arrays to Matlab and vice versa which shows

    shape = cellfun(@int64,cell(myfunction_results.shape));
ls = py.array.array('d',myfunction_results.flatten('F').tolist());
p = double(ls);

    But I must say that is very cumbersome....is there an easier way?

  • Writing to Text Files as a Struct vs.Cell Array

    So I am trying to write data to a .txt file and I have encountered some difficulties. My understanding is that if I want to write struct data to a text file I need to first convert it to a table using struct2table and then use writetable to write it to the file. When I do this, however, the data is comma delimited in the text file. I really like the table format as it appears in MATLAB, but I can't find a way to make it appear that way in the text file (I assume that Excel reads in data that is comma delimited, which is why the data is formatted that way). Now, if I format the data as a cell array and then write that to the .txt file, then it looks great; however, the struct format is nice in that I can access data points easier and plot the data. So, I am a little lost on what route I should take to solve this problem. One idea I had was to format the data as a struct array and then when I want to write it to a .txt convert it to a cell array. The other idea I had was to somehow manipulate the data format when I use writetable. I can, for instance, change the delimiter to a tab and that looks great, except that the data does not line up (e.g. if I have Frequency and Power as column headers, the numbers below them in the table are not aligned with those headers). This, of course, is trivial if using `fprintf'; but I can't use that on a struct array (from what I understand).

    I hope this is comprehensive enough, but if there is anything else I can provide please let me know and thank you in advance.

  • Calling back up to a certain line in a certain file, for each buttons in MATLAB gui (guide)

    I recently discovered about MATLAB's builtin guide GUI interface. I wanted to make an app to run some example model codes that I have. I will list them in order below to avoid possible confusion!

    1. A button, once pressed, runs up to line 50 in file A, which creates data.
    2. This new data displayed to the table.
    3. A button that runs up to line 100 in file A, which plots two different graphs.
    4. A slider that changes a parameter of the plot function which would then change the shape of the plot.

    I'm just really new to using the guide, so I'm kind of lost on how to start, so any help/pointers would be appreciated!!

    Also, is MATLAB's guide a popular way of creating apps/interfaces for the code, or is there a better alternative? I kind of find the resources/help guides limited for it.

  • Implementing an id3 algorithm - error in 2 functions

    I would like to make a ID3 algorithm but I am running into problems here.

    First of all I would like to make 2 functions.

    The first one is the following:

    def func1_learn(X, y, impurity_measure = "entropy"):
      impurity_measure = "entropy"

    this function should learn a decision tree with entropy as impurity_measure so if I wanted to call on the func1_learn function by using:

    func1_learn(X, y, impurity_measure = "entropy")

    or like this:

    func1_learn(X, y)

    then in both cases, the function should learn a decision tree with entropy as impurity measure. But my code doesn´t do that, I just get the wrong output.

    My second function should predict class label of some new data point x.

    def func2_predict(x, tree):
    predictions = classifier.predict(x)
    #predict class label of some new data point x

    This function should predict class label of some new data point x.

    My question is, How do I change the code for the functions so that they work as desired above? What mistake did I do?

  • How to return transformed data from an ML.Net pipeline before a predictor is applied

    Here is the creation of the ML.Net pipeline object copied from the TaxiFarePrediction example.

            LearningPipeline pipeline = new LearningPipeline
        {
            new TextLoader(TrainDataPath).CreateFrom<TaxiTrip>(separator:','),
            new ColumnCopier(("FareAmount", "Label")),
            new CategoricalOneHotVectorizer("VendorId","RateCode","PaymentType"),
            new ColumnConcatenator("Features","VendorId","RateCode","PassengerCount","TripDistance","PaymentType"),
            new FastTreeRegressor()
        };

    Essentially, I'd like to return the data after the ColumnCopier, the CategoricalOneHotVectorizer and the ColumnConcatenator have been applied.

  • Implement the ID3 algorithm from scratch

    I would like to make a ID3 algorithm but I am running into problems here.

    First of all I would like to make 2 functions.

    The first one is the following:

    def func1_learn(X, y, impurity_measure = "entropy"):
      impurity_measure = "entropy"

    this function should learn a decision tree with entropy as impurity_measure so if I wanted to call on the func1_learn function by using:

    func1_learn(X, y, impurity_measure = "entropy")

    or like this:

    func1_learn(X, y)

    then in both cases, the function should learn a decision tree with entropy as impurity measure. But my code doesn´t do that, I just get the wrong output.

    My second function should predict class label of some new data point x.

    def func2_predict(x, tree):
    #predict class label of some new data point x

    This function should predict class label of some new data point x.

    My question is: How do I write the functions to do as I described above? How do I do this?

  • replace manual threads and queue by monitoredsession 
    coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)

    Can anyone tell how exactly to replace above two line of code with tf.train.MonitoredSession and what effects the change has?

  • how to merge different dimensions arrays in python?

    I am analyzing some image represented datasets using keras. I am stuck that I have two different dimensions of images. Please see the snapshot. Features has 14637 images having dimension (10,10,3) and features2 has dimension (10,10,100)

    enter image description here

    Is there any way that I can merge/concatenate these two data together.?

  • multi label classification in keras

    I tried to build a model that would help me identify images of a multi label classification problem, for example if I had pictures of cats, dogs and cows. I ran a CNN model but it didnt catch at all (gave a precision of 33%). Can anyone please share a model that works (even if the accuracy is just reasonable)? Thanks in advance to everyone! [attached a my code mentioned above]

    from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, 
BatchNormalization
from keras.callbacks import LearningRateScheduler
from keras.optimizers import adam, SGD
from keras.preprocessing.image import ImageDataGenerator
from keras.applications.vgg16 import VGG16


# 2 - Create network layers
image_width = 200
image_height = 200

model = Sequential()
model.add(Conv2D(filters=16, kernel_size=(3,3), 
activation='relu',input_shape=( 
(image_width,image_height,3)))
model.add(BatchNormalization())
model.add(Conv2D(filters=16, kernel_size=(3,3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPool2D(strides=(2,2)))
model.add(Dropout(0.25))
# Stage II = make it more compex with 'filters = 32'
model.add(Conv2D(filters=32, kernel_size=(3,3), activation='relu'))
model.add(BatchNormalization())
model.add(Conv2D(filters=32, kernel_size=(3,3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPool2D(strides=(2,2)))

model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(1024, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax'))


# We'll Randomize the training set (shuffle), to avoid overfitting 
(augmentation)
datagen = ImageDataGenerator(zoom_range = 0.1,
                        height_shift_range = 0.1,
                        width_shift_range = 0.1,
                        rotation_range = 10)

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=

    ['accuracy'])

    # automatically retrieve images and their classes for train and validation 
train_generator = datagen.flow_from_directory(
    train_dataset,
    target_size=(image_width, image_height),
    batch_size=32,
    class_mode='categorical')


validation_generator = datagen.flow_from_directory(
    validation_dataset,
    target_size=(image_width, image_height),
    batch_size=32,
    class_mode='categorical')

# Now let's fit the model on the validation set

model.fit_generator(
    train_generator,
    steps_per_epoch=50,
    epochs=500,
    validation_data=validation_generator,
    validation_steps=15)
  • Error when checking input: expected flatten_input to have 3 dimensions, but got array with shape (None, 100, 100, 1)

    Using TensorFlow/Keras, I want to classify pictures into two classes, selfie and non-selfie.

    I have gathered samples into two filesystem folders, one for each category.

    I implemented the training below by following the official tutorial for MNIST fashion (which is also a pictures classification problem), after using loading pictures from the filesystem as seen at https://stackoverflow.com/a/52417770/226958.

    Unfortunately, I get an error:

    1.10.1
Tensor("IteratorGetNext:0", shape=(?, 100, 100, 1), dtype=float32)
Tensor("IteratorGetNext:1", shape=(?,), dtype=int32)
Traceback (most recent call last):
  File "run.py", line 50, in <module>
    model.fit(images, labels, epochs=1, steps_per_epoch=60000)
  File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training.py", line 1278, in fit
    validation_split=validation_split)
  File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training.py", line 878, in _standardize_user_data
    exception_prefix='input')
  File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training_utils.py", line 182, in standardize_input_data
    'with shape ' + str(data_shape))
ValueError: Error when checking input: expected flatten_input to have 3 dimensions, but got array with shape (None, 100, 100, 1)

    Here is the source code:

    import tensorflow as tf
print(tf.__version__)

out_shape = tf.convert_to_tensor([100, 100])
batch_size = 2

image_paths, labels = ["selfies-data/1", "selfies-data/2"], [1, 2]
epoch_size = len(image_paths)
image_paths = tf.convert_to_tensor(image_paths, dtype=tf.string)
labels = tf.convert_to_tensor(labels)

# The images loading part is from https://stackoverflow.com/a/52417770/226958
dataset = tf.data.Dataset.from_tensor_slices((image_paths, labels))
dataset = dataset.repeat().shuffle(epoch_size)

def map_fn(path, label):
    # path/label represent values for a single example
    image = tf.image.decode_jpeg(tf.read_file(path))

    # some mapping to constant size - be careful with distorting aspec ratios
    image = tf.image.resize_images(image, out_shape)
    image = tf.image.rgb_to_grayscale(image)
    # color normalization - just an example
    image = tf.to_float(image) * (2. / 255) - 1
    return image, label

# num_parallel_calls > 1 induces intra-batch shuffling
dataset = dataset.map(map_fn, num_parallel_calls=8)
dataset = dataset.batch(batch_size)
dataset = dataset.prefetch(1)

images, labels = dataset.make_one_shot_iterator().get_next()

# All of the following is from https://www.tensorflow.org/tutorials/keras/basic_classification
from tensorflow import keras

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(100, 100)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.softmax)
])

model.compile(optimizer=tf.train.AdamOptimizer(),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

print(images)
print(labels)
model.fit(images, labels, epochs=epoch_size, steps_per_epoch=60000)

    While there are similar questions which I have read, I don't see any question with this None.

    How can I adapt Keras to my input, or transform my input so that Keras accepts it?

  • resnet50 - increasing training speed? keras

    hth do I increase the speed of this? I mean the loss is moving down by hairs. HAIRS.

    Epoch 1/30
4998/4998 [==============================] - 307s 62ms/step - loss: 0.6861 - acc: 0.6347
Epoch 2/30
4998/4998 [==============================] - 316s 63ms/step - loss: 0.6751 - acc: 0.6387
Epoch 3/30
4998/4998 [==============================] - 357s 71ms/step - loss: 0.6676 - acc: 0.6387
Epoch 4/30
4998/4998 [==============================] - 376s 75ms/step - loss: 0.6625 - acc: 0.6387
Epoch 5/30
4998/4998 [==============================] - 354s 71ms/step - loss: 0.6592 - acc: 0.6387
Epoch 6/30
4998/4998 [==============================] - 345s 69ms/step - loss: 0.6571 - acc: 0.6387
Epoch 7/30
4998/4998 [==============================] - 349s 70ms/step - loss: 0.6559 - acc: 0.6387

    Model Architecture: resnet50 (CNN with skip connections)

    Except instead of 1 FC I have two. And I changed the softmax output to sigmoid for binary classification.

    num positive training data: 1806 num neg training data: 3192

    My output is represented by a 1 or 0 for each example ( [0, 0, 1, 1, ...])

    batches = 40, num epochs =30, but that doesn't matter because the loss stopped

  • unstable loss and accuracy?

    My setup in keras with all the parameters with resenet 50 architecture, softmax activation exchanged with sigmoid for binary classification :

    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model.fit(train_x, train_y, epochs = 40, batch_size = 32)

    My results:

    Epoch 1/40
4998/4998 [==============================] - 328s 66ms/step - loss: 0.9148 - acc: 0.6246
Epoch 2/40
4998/4998 [==============================] - 360s 72ms/step - loss: 0.7279 - acc: 0.6369
Epoch 3/40
4998/4998 [==============================] - 368s 74ms/step - loss: 0.6633 - acc: 0.6705
Epoch 4/40
4998/4998 [==============================] - 349s 70ms/step - loss: 0.5865 - acc: 0.7495
Epoch 5/40
4998/4998 [==============================] - 387s 77ms/step - loss: 0.5267 - acc: 0.7801
Epoch 6/40
4998/4998 [==============================] - 3124s 625ms/step - loss: 0.5106 - acc: 0.8015
Epoch 7/40
4998/4998 [==============================] - 312s 62ms/step - loss: 0.4953 - acc: 0.8055
Epoch 8/40
4998/4998 [==============================] - 355s 71ms/step - loss: 0.7689 - acc: 0.6705
Epoch 9/40
4998/4998 [==============================] - 349s 70ms/step - loss: 0.6370 - acc: 0.7063
Epoch 10/40
4998/4998 [==============================] - 353s 71ms/step - loss: 0.5351 - acc: 0.7699
Epoch 11/40
4998/4998 [==============================] - 363s 73ms/step - loss: 0.5274 - acc: 0.7991
Epoch 12/40
4998/4998 [==============================] - 361s 72ms/step - loss: 0.5707 - acc: 0.7701
Epoch 13/40
4998/4998 [==============================] - 348s 70ms/step - loss: 0.6168 - acc: 0.7193
Epoch 14/40
4998/4998 [==============================] - 357s 71ms/step - loss: 0.7444 - acc: 0.6459
Epoch 15/40
4998/4998 [==============================] - 356s 71ms/step - loss: 0.6682 - acc: 0.6847
Epoch 16/40
4998/4998 [==============================] - 356s 71ms/step - loss: 0.6614 - acc: 0.6653
Epoch 17/40
4998/4998 [==============================] - 343s 69ms/step - loss: 0.6009 - acc: 0.7153
Epoch 18/40
4998/4998 [==============================] - 346s 69ms/step - loss: 0.6060 - acc: 0.7277
Epoch 19/40
4998/4998 [==============================] - 340s 68ms/step - loss: 0.7800 - acc: 0.6325
Epoch 20/40
4998/4998 [==============================] - 343s 69ms/step - loss: 0.7405 - acc: 0.6659
Epoch 21/40
4998/4998 [==============================] - 341s 68ms/step - loss: 0.8655 - acc: 0.6557
Epoch 22/40
4998/4998 [==============================] - 334s 67ms/step - loss: 0.7600 - acc: 0.6641
Epoch 23/40

    Why the heck is the loss and accuracy wobbling back and fourth like that?

    num positive training data 1806 num neg training data 3192

    My output is represented by a 1 or 0 for each example ( [0, 0, 1, 1, ...])

    Update: I got stability by adding another dense layer, but now it moves too slow. I read some place that instability can mean not enough data.

    I assume down votes don't mean it's a bad question. I assume it means the person down voting it doesn't know enough about machine learning to perform analysis or have anything of value to add to the discussion.

  • Select attribute in weka ends up 0 for all attributes

    I have a data set with 46 attributes and 74170 instances in weka. I suppose I need to select best attributes for better accuracy. first I tried to select attributes using filter with CfsSubsetEval and BestFirst for search, So weka decreased my attributes from 47 to 17 number. Then I used Select Attributes tab for testing my data accuracy and error rate, I used WrapperSubsetEval for evaluator and Adaboost for classifier and naive Bayes for base learner and BestFirst for search and 10 fold cross validation for training. but at the end all the attributes get '0'. I am not sure what is wrong.

  • How to use ElasticNet with AdaBoostClassifier?

    Is it possible to use ElasticNet with AdaBoostClassifier in sklearn?

    from sklearn import ensemble
from sklearn.linear_model import ElasticNet

enet = ElasticNet(alpha = 0.0001, l1_ratio = 0.5)
clf5 = ensemble.AdaBoostClassifier(enet, n_estimators=10)
clf5 = clf5.fit(X_train, Y_train)

    This results in

    TypeError: AdaBoostClassifier with algorithm='SAMME.R' requires that the weak learner supports the calculation of class probabilities with a predict_proba method. Please change the base estimator or set algorithm='SAMME' instead.

  • opencv_traincascade stopped at stage 1

    I use my own data to train classifier by opencv_traincascade. The number of positive and negative samples are big enough and the ratio is about 1:3. However, the code only train stage 0. Is there any one could help me?

    the training process:

    image