outofsample data classification
A question from an online course:
A difficulty that arises from trying to classify outofsample data is that the actual classification may not be known, therefore making it hard to produce an accurate result.
True or False?
See also questions close to this topic

Implementing an id3 algorithm  error in 2 functions
I would like to make a ID3 algorithm but I am running into problems here.
First of all I would like to make 2 functions.
The first one is the following:
def func1_learn(X, y, impurity_measure = "entropy"): impurity_measure = "entropy"
this function should learn a decision tree with entropy as impurity_measure so if I wanted to call on the func1_learn function by using:
func1_learn(X, y, impurity_measure = "entropy")
or like this:
func1_learn(X, y)
then in both cases, the function should learn a decision tree with entropy as impurity measure. But my code doesn´t do that, I just get the wrong output.
My second function should predict class label of some new data point x.
def func2_predict(x, tree): predictions = classifier.predict(x) #predict class label of some new data point x
This function should predict class label of some new data point x.
My question is, How do I change the code for the functions so that they work as desired above? What mistake did I do?

How to return transformed data from an ML.Net pipeline before a predictor is applied
Here is the creation of the ML.Net pipeline object copied from the TaxiFarePrediction example.
LearningPipeline pipeline = new LearningPipeline { new TextLoader(TrainDataPath).CreateFrom<TaxiTrip>(separator:','), new ColumnCopier(("FareAmount", "Label")), new CategoricalOneHotVectorizer("VendorId","RateCode","PaymentType"), new ColumnConcatenator("Features","VendorId","RateCode","PassengerCount","TripDistance","PaymentType"), new FastTreeRegressor() };
Essentially, I'd like to return the data after the ColumnCopier, the CategoricalOneHotVectorizer and the ColumnConcatenator have been applied.

Implement the ID3 algorithm from scratch
I would like to make a ID3 algorithm but I am running into problems here.
First of all I would like to make 2 functions.
The first one is the following:
def func1_learn(X, y, impurity_measure = "entropy"): impurity_measure = "entropy"
this function should learn a decision tree with entropy as impurity_measure so if I wanted to call on the func1_learn function by using:
func1_learn(X, y, impurity_measure = "entropy")
or like this:
func1_learn(X, y)
then in both cases, the function should learn a decision tree with entropy as impurity measure. But my code doesn´t do that, I just get the wrong output.
My second function should predict class label of some new data point x.
def func2_predict(x, tree): #predict class label of some new data point x
This function should predict class label of some new data point x.
My question is: How do I write the functions to do as I described above? How do I do this?

Error when checking input: expected flatten_input to have 3 dimensions, but got array with shape (None, 100, 100, 1)
Using TensorFlow/Keras, I want to classify pictures into two classes, selfie and nonselfie.
I have gathered samples into two filesystem folders, one for each category.
I implemented the training below by following the official tutorial for MNIST fashion (which is also a pictures classification problem), after using loading pictures from the filesystem as seen at https://stackoverflow.com/a/52417770/226958.
Unfortunately, I get an error:
1.10.1 Tensor("IteratorGetNext:0", shape=(?, 100, 100, 1), dtype=float32) Tensor("IteratorGetNext:1", shape=(?,), dtype=int32) Traceback (most recent call last): File "run.py", line 50, in <module> model.fit(images, labels, epochs=1, steps_per_epoch=60000) File "/home/nico/.local/lib/python2.7/sitepackages/tensorflow/python/keras/engine/training.py", line 1278, in fit validation_split=validation_split) File "/home/nico/.local/lib/python2.7/sitepackages/tensorflow/python/keras/engine/training.py", line 878, in _standardize_user_data exception_prefix='input') File "/home/nico/.local/lib/python2.7/sitepackages/tensorflow/python/keras/engine/training_utils.py", line 182, in standardize_input_data 'with shape ' + str(data_shape)) ValueError: Error when checking input: expected flatten_input to have 3 dimensions, but got array with shape (None, 100, 100, 1)
Here is the source code:
import tensorflow as tf print(tf.__version__) out_shape = tf.convert_to_tensor([100, 100]) batch_size = 2 image_paths, labels = ["selfiesdata/1", "selfiesdata/2"], [1, 2] epoch_size = len(image_paths) image_paths = tf.convert_to_tensor(image_paths, dtype=tf.string) labels = tf.convert_to_tensor(labels) # The images loading part is from https://stackoverflow.com/a/52417770/226958 dataset = tf.data.Dataset.from_tensor_slices((image_paths, labels)) dataset = dataset.repeat().shuffle(epoch_size) def map_fn(path, label): # path/label represent values for a single example image = tf.image.decode_jpeg(tf.read_file(path)) # some mapping to constant size  be careful with distorting aspec ratios image = tf.image.resize_images(image, out_shape) image = tf.image.rgb_to_grayscale(image) # color normalization  just an example image = tf.to_float(image) * (2. / 255)  1 return image, label # num_parallel_calls > 1 induces intrabatch shuffling dataset = dataset.map(map_fn, num_parallel_calls=8) dataset = dataset.batch(batch_size) dataset = dataset.prefetch(1) images, labels = dataset.make_one_shot_iterator().get_next() # All of the following is from https://www.tensorflow.org/tutorials/keras/basic_classification from tensorflow import keras model = keras.Sequential([ keras.layers.Flatten(input_shape=(100, 100)), keras.layers.Dense(128, activation=tf.nn.relu), keras.layers.Dense(10, activation=tf.nn.softmax) ]) model.compile(optimizer=tf.train.AdamOptimizer(), loss='sparse_categorical_crossentropy', metrics=['accuracy']) print(images) print(labels) model.fit(images, labels, epochs=epoch_size, steps_per_epoch=60000)
While there are similar questions which I have read, I don't see any question with this
None
.How can I adapt Keras to my input, or transform my input so that Keras accepts it?

resnet50  increasing training speed? keras
hth do I increase the speed of this? I mean the loss is moving down by hairs. HAIRS.
Epoch 1/30 4998/4998 [==============================]  307s 62ms/step  loss: 0.6861  acc: 0.6347 Epoch 2/30 4998/4998 [==============================]  316s 63ms/step  loss: 0.6751  acc: 0.6387 Epoch 3/30 4998/4998 [==============================]  357s 71ms/step  loss: 0.6676  acc: 0.6387 Epoch 4/30 4998/4998 [==============================]  376s 75ms/step  loss: 0.6625  acc: 0.6387 Epoch 5/30 4998/4998 [==============================]  354s 71ms/step  loss: 0.6592  acc: 0.6387 Epoch 6/30 4998/4998 [==============================]  345s 69ms/step  loss: 0.6571  acc: 0.6387 Epoch 7/30 4998/4998 [==============================]  349s 70ms/step  loss: 0.6559  acc: 0.6387
Model Architecture: resnet50 (CNN with skip connections)
Except instead of 1 FC I have two. And I changed the softmax output to sigmoid for binary classification.
num positive training data: 1806 num neg training data: 3192
My output is represented by a 1 or 0 for each example ( [0, 0, 1, 1, ...])
batches = 40, num epochs =30, but that doesn't matter because the loss stopped

unstable loss and accuracy?
My setup in keras with all the parameters with resenet 50 architecture, softmax activation exchanged with sigmoid for binary classification :
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) model.fit(train_x, train_y, epochs = 40, batch_size = 32)
My results:
Epoch 1/40 4998/4998 [==============================]  328s 66ms/step  loss: 0.9148  acc: 0.6246 Epoch 2/40 4998/4998 [==============================]  360s 72ms/step  loss: 0.7279  acc: 0.6369 Epoch 3/40 4998/4998 [==============================]  368s 74ms/step  loss: 0.6633  acc: 0.6705 Epoch 4/40 4998/4998 [==============================]  349s 70ms/step  loss: 0.5865  acc: 0.7495 Epoch 5/40 4998/4998 [==============================]  387s 77ms/step  loss: 0.5267  acc: 0.7801 Epoch 6/40 4998/4998 [==============================]  3124s 625ms/step  loss: 0.5106  acc: 0.8015 Epoch 7/40 4998/4998 [==============================]  312s 62ms/step  loss: 0.4953  acc: 0.8055 Epoch 8/40 4998/4998 [==============================]  355s 71ms/step  loss: 0.7689  acc: 0.6705 Epoch 9/40 4998/4998 [==============================]  349s 70ms/step  loss: 0.6370  acc: 0.7063 Epoch 10/40 4998/4998 [==============================]  353s 71ms/step  loss: 0.5351  acc: 0.7699 Epoch 11/40 4998/4998 [==============================]  363s 73ms/step  loss: 0.5274  acc: 0.7991 Epoch 12/40 4998/4998 [==============================]  361s 72ms/step  loss: 0.5707  acc: 0.7701 Epoch 13/40 4998/4998 [==============================]  348s 70ms/step  loss: 0.6168  acc: 0.7193 Epoch 14/40 4998/4998 [==============================]  357s 71ms/step  loss: 0.7444  acc: 0.6459 Epoch 15/40 4998/4998 [==============================]  356s 71ms/step  loss: 0.6682  acc: 0.6847 Epoch 16/40 4998/4998 [==============================]  356s 71ms/step  loss: 0.6614  acc: 0.6653 Epoch 17/40 4998/4998 [==============================]  343s 69ms/step  loss: 0.6009  acc: 0.7153 Epoch 18/40 4998/4998 [==============================]  346s 69ms/step  loss: 0.6060  acc: 0.7277 Epoch 19/40 4998/4998 [==============================]  340s 68ms/step  loss: 0.7800  acc: 0.6325 Epoch 20/40 4998/4998 [==============================]  343s 69ms/step  loss: 0.7405  acc: 0.6659 Epoch 21/40 4998/4998 [==============================]  341s 68ms/step  loss: 0.8655  acc: 0.6557 Epoch 22/40 4998/4998 [==============================]  334s 67ms/step  loss: 0.7600  acc: 0.6641 Epoch 23/40
Why the heck is the loss and accuracy wobbling back and fourth like that?
num positive training data 1806 num neg training data 3192
My output is represented by a 1 or 0 for each example ( [0, 0, 1, 1, ...])
Update: I got stability by adding another dense layer, but now it moves too slow. I read some place that instability can mean not enough data.
I assume down votes don't mean it's a bad question. I assume it means the person down voting it doesn't know enough about machine learning to perform analysis or have anything of value to add to the discussion.