How to use embedding to onehot encode before passing
I am able to train my seq2seq model when onehot encodded input is passed in the fit function. How would I achieve the same thing if input is not one hot encoded?
Following code works
def seqModel():
latent_dim = 256 # Latent dimensionality of the encoding space.
encoder_inputs = Input(shape=(None, input_vocab_size))
decoder_inputs = Input(shape=(None, output_vocab_size))
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
encoder_states = [state_h, state_c]
decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs,\
initial_state=encoder_states)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
return model
def train(data):
model = seqModel()
#compile and get data
model.fit([to_one_hot(input_texts, num_encoder_tokens),to_one_hot(target_text, num_decoder_tokens)], outputs,batch_size=3, epochs=5)
I am asked not to onehot encode in the train method. How would I do it in the seqModel method? Is Embedding right way to onehot encode?
See also questions close to this topic

Rtask on Boston dataset part of MASS library in R. Task: Plot accuracy of KNN, K=1:50.
I have done the following but it gives me an error saying NAs introduced by coercion
ourBoston=data.frame(Boston) ourBoston$high.medv=NA levels(ourBoston$high.medv)=c("no","yes") ourBoston$high.medv[Boston$medv>25]<"yes" ourBoston$high.medv[Boston$medv<=25]<"no" result<rep(0,50) for (i in 1:50) { result=knn(train=data.frame(ourBoston$lstat),test=data.frame(ourBoston$lstat),cl=ourBoston$high.medv,k=i) result[i]=sum(as.integer(ourBoston$high.medv))/length(result) } qplot(1:50,result[1:50])
I have created a column and added it to a new data frame>ourBoston
The lstat variable is the best predictor I have to select for response variable high.medv.
The training and test data set have to be the same.
On running the code I got 50 warnings as: In as.integer(ourBoston$high.medv) : NAs introduced by coercion

Error in data.frame(ID = sales$Store, Date = sales$Date, Prediction = test_sales) : arguments imply differing number of rows: 27320, 27214
I am working on a dataset to forecast the sales. I have Store ID(50 stores), Date(2016,2017) and sales. So, I need to forecast the sales for 2018.
library(forecast) library(fma) library(dplyr) library(ggplot2) #visualizations library(gridExtra) #viewing multiple plots together library(tidytext) #text mining library(wordcloud2) library(caret) library(lattice) sales_2016 = read.csv("C:/Users/manjeet.singh/Desktop/2016XP.csv") sales_2017 = read.csv("C:/Users/manjeet.singh/Desktop/2017XP.csv") sales = rbind(sales_2016,sales_2017) preProcValues < preProcess(sales, method = c("knnImpute","center","scale")) train_processed < predict(preProcValues, sales) sum(is.na(sales)) sales$Date < as.Date(sales$Date,format= "%Y%m%d") tsDataSales<ts(sales$Sales ,start=c(2016,1),frequency=52) plot(tsDataSales) plot(decompose(tsDataSales)) train_sales<window(tsDataSales,start=c(2016,1),end=c(2017,52),frequency = 52) test_sales<window(tsDataSales,start=2018,frequency = 52) ndiffs(train_sales) Acf(train_sales,lag.max=52,plot=TRUE) Pacf(train_sales,lag.max=52,plot=TRUE,main="Original Time Series") tst_arima < arima(train_sales,order=c(1,1,1)) summary(tst_arima) tst_arima_resi < residuals(tst_arima) summary(tst_arima_resi) plot(tst_arima_resi) qqnorm(tst_arima_resi) qqline(tst_arima_resi) box<Box.test(tst_arima_resi,lag=52,type="LjungBox",fitdf=1) box forecastval < forecast(tst_arima,h=52) plot(forecastval,main="Prediction from Auto Arima for Weekly Sales") lines(test_sales,col="green") **result = data.frame("ID"=sales$Store,"Date"= sales$Date, "Prediction"= test_sales)**
I am getting this error when I am trying the write the CSV file.
Error in data.frame(ID = sales$Store, Date = sales$Date, Prediction = test_sales) : arguments imply differing number of rows: 27320, 27214
Can you please tell me where I am going wrong.
I have tried everything here but still struggling.

Dataset required to train chatbot
I want huge amount of data to train my chatbot. All coding is done in python. Already having database linked with sql.
**Machine learning **
Tensorflow
Neural networking

Can I use the output of a model's layer as a target to train that same model?
Let's say I have a model with one input and two outputs. And I want the output of the third layer of my model to be the y_true in my cost function for my second output. I've tried this:
model.fit(x, [y, model.layers[3].output], ...)
But got the error:
'Tensor' object has no attribute 'ndim'
Which I believe is referring to the second y_true I gave the fit method.
Is it possible to do something like this in Keras? If so, how?

How do you train a neural network without an exact answer?
Most neural networks use backpropagation to learn, but from how I've understood it you need an exact answer to what the outputs should be for this to work. What I want to do is to learn a walker bot to walk, and have a score or fitness variable to evaluate it. Any ideas on how you could do this in for example python or keras?

How to use weigths extracted from a LSTM recurrent neural network
I have trained an LSTM recurrent neural network for sequence (time series) classification with Keras in Python.
Features are collated in a shape (batch_size, timesteps, data_dim). My training examples are 1000 in total. The final goal is to have a classification among 5 classes. Here is a snippet of my code.
#defining some model features data_dim = 15 timesteps = 20 num_classes = len(one_hot_train_labels[1,:]) batch_size = len(ytrain) #reshaping array for LSTM training xtrain=numpy.reshape(xtrain, (len(ytrain), timesteps, data_dim)) xtest=numpy.reshape(xtest, (len(ytest), timesteps, data_dim)) rms = optimizers.RMSprop(lr=0.001, rho=0.9, epsilon=None, decay=0.0) #It is recommended to leave the parameters #of this optimizer at their default values (except the learning rate, which can be freely tuned). # create the model model = Sequential() model.add(LSTM(101, dropout=0.5, recurrent_dropout=0.5, input_shape=(timesteps, data_dim), activation='tanh')) model.add(Dense(5, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer=rms, metrics=['accuracy']) print(model.summary()) history = model.fit(xtrain, one_hot_train_labels, epochs=200, batch_size=10) # Final evaluation of the model scores = model.evaluate(xtrain, one_hot_train_labels, verbose=0) print("Accuracy: %.2f%%" % (scores[1]*100)) scores = model.evaluate(xtest, one_hot_test_labels, verbose=0) print("Accuracy: %.2f%%" % (scores[1]*100))
Since I want to use and implement the classifier elsewhere, I have extracted the weights using:
weights = [model.get_weights() for layer in model.layers]
Having used traditional neural networks and logistic regression in the past, I was expecting 2 matrices for each layer, one with the polynomial weights and one with the bias units, and then using the activation function (in this case tanh and softmax function) to progressively find the probabilities of falling in one of the 5 classes.
But I am now confused because calling the weights returns me 5 matrices with the following sizes:
 (15, 400)
 (100, 400)
 (400,)
 (100,5)
 (5,)
Now, I understand the LSTM works with 4 different blocks:
 input from vector
 memory from previous block
 memory from current block
 output from previous block
and hence why I have a size of 400 for the 2n dimension of my matrices.
And now my questions:
How can I use those weights in a cascade way to eventually get the class probabilities, using the activation functions (as in traditional neural networks)?
where is the bias unit for the input layer?
Thanks to everyone can help to clarify and help in understanding how to use this powerful tool as LSTM networks.
Hope this would be helpful not just for me.

Error in reshaping input tokenized text predicting the sentiments in a lstm rnn
I am new to neural network and have been learning it's application in the field of text analytics, so i have used a lstm rnn for the application in python.
After training the model on a dataset of dimension 20,000*1 (2000being the text and ,1being the sentiment of the text) i got a good accuracy of 99%, after which i validated the model which was working fine(using the model.predict()function).
Now just to test my model i have been trying to give random text inputs either from a dataframe or variables containing some text but i always landup with the error of reshaping the array , where it is required that the input to the rnn model be of the dimension (1,30).
But when i reinput the training data into the model for prediction , the model works absolutely fine , why is this happening?
link for the screenshot of error
link for image of model summary
I am just stuck here and any kind of suggestion will help me learning more about rnn, i am attaching the error and the rnn model code with this request.
Thank You
Regards
Tushar Upadhyay
import numpy as np import pandas as pd import keras import sklearn from sklearn.feature_extraction.text import CountVectorizer from keras.preprocessing.text import Tokenizer from keras.preprocessing.sequence import pad_sequences from keras.models import Sequential from keras.layers import Dense, Embedding, LSTM from sklearn.model_selection import train_test_split from keras.utils.np_utils import to_categorical import re data=pd.read_csv('..../twitter_tushar_data.csv') max_fatures = 4000 tokenizer = Tokenizer(num_words=max_fatures, split=' ') tokenizer.fit_on_texts(data['tweetText'].values) X = tokenizer.texts_to_sequences(data['tweetText'].values) X = pad_sequences(X) embed_dim = 128 lstm_out = 196 model = Sequential() keras.layers.core.SpatialDropout1D(0.2) #used to avoid overfitting model.add(Embedding(max_fatures, embed_dim,input_length = X.shape[1])) model.add(LSTM(196, recurrent_dropout=0.2, dropout=0.2)) model.add(Dense(2,activation='softmax')) model.compile(loss = 'categorical_crossentropy', optimizer='adam',metrics = ['accuracy']) print(model.summary()) #splitting data in training and testing parts Y = pd.get_dummies(data['SA']).values X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.30, random_state = 42) print(X_train.shape,Y_train.shape) print(X_test.shape,Y_test.shape) batch_size = 128 model.fit(X_train, Y_train, epochs = 7, batch_size=batch_size, verbose = 2) validation_size = 3500 X_validate = X_test[validation_size:] Y_validate = Y_test[validation_size:] X_test = X_test[:validation_size] Y_test = Y_test[:validation_size] score,acc = model.evaluate(X_test, Y_test, verbose = 2, batch_size = 128) print("score: %.2f" % (score)) print("acc: %.2f" % (acc)) pos_cnt, neg_cnt, pos_correct, neg_correct = 0, 0, 0, 0 for x in range(len(X_validate)): result = model.predict(X_validate[x].reshape(1,X_test.shape[1]),batch_size=1,verbose = 2)[0] if np.argmax(result) == np.argmax(Y_validate[x]): if np.argmax(Y_validate[x]) == 0: neg_correct += 1 else: pos_correct += 1 if np.argmax(Y_validate[x]) == 0: neg_cnt += 1 else: pos_cnt += 1 print("pos_acc", pos_correct/pos_cnt*100, "%") print("neg_acc", neg_correct/neg_cnt*100, "%")

Issue with Keras Dense outputting multitimestep, multvariable
So I am training my model with input of (6000, 4, 2) (6000 samples, 4 features, 2 timesteps) and ideal I would have an output of (6000, 2, 3) (6000 samples, 2 features, 3 timesteps). Currently I can only get it to work with an output of (6000, 3) with a Dense(3) at the end, but I can't figure out how to use Dense for my ideal output.
So my first approach was to concatenate 2 Dense(3) networks as shown below, but this doesn't seem to work:
model = Sequential() model.add(LSTM(25, batch_input_shape=(1, 4, 2), stateful=True)) model_1 = Sequential() model_1.add(Dense(3, input_shape=(1, 25))) model_2 = Sequential model_1.add(Dense(3, input_shape=(1, 25))) model.add(Merge([model_1, model_2], mode='concat')) model.compile(loss='mean_squared_error', optimizer='adam') for _ in range(epochs): model.fit(data, [data_ans1, data_ans2], epochs=1, batch_size=n_batch, verbose=2, shuffle=False) model.reset_states()
The error is on the line:
model.add(Concatenate([model_1, model_2]))
The error says:
get_output_shape_at() missing 1 required positional argument: 'node_index'
Can anyone see anything notably wrong in this, or even better, does anyone have a better solution to get a multitimestep, multifeature Dense output?

Extracting weights from an LSTM neural network using Keras
I have trained a recurrent neural network (LSTM) in keras but now I am struggling to put all the pieces together. Specifically, I cannot understand how to recompose the matrices of weights.
I have one input, one hidden and one output layer, as follows:
# create the model model = Sequential() model.add(LSTM(100, dropout=0.5, recurrent_dropout=0.5, input_shape=(timesteps, data_dim), activation='tanh')) model.add(Dense(5, activation='softmax'))
when I call
model.get_weights()
I only get the bias units for the hidden and output layers but the ones for the input layers appear missing. Namely, I have a 15x400 matrix, then a 400x101 and a 101X5.Is there something that I am missing out here?
Thanks for your help

LabelBinarizer yields different result in multiclass example
When executing the multiclass example in the scikitlearn tutorial
I came across a slight oddity.
>>> import sklearn >>> sklearn.__version__ 0.19.1 >>> from sklearn.svm import SVC >>> from sklearn.multiclass import OneVsRestClassifier >>> from sklearn.preprocessing import LabelBinarizer >>> X = [[1, 2], [2, 4], [4, 5], [3, 2], [3, 1]] >>> y = [0, 0, 1, 1, 2] # Three classes >>> clf = OneVsRestClassifier(estimator=SVC(random_state=0)) >>> clf.fit(X, y).predict(X) array([0, 0, 1, 1, 2])
This is all fine. Now with onehot encoding:
>>> y = LabelBinarizer().fit_transform(y) >>> y array([[1, 0, 0], [1, 0, 0], [0, 1, 0], [0, 1, 0], [0, 0, 1]])
I would expect the label binarizer to only encode the target, but not having an influence on the classifier. However it yields a different result:
>>> clf.fit(X, y).predict(X) array([[1, 0, 0], [1, 0, 0], [0, 1, 0], [0, 0, 0], [0, 0, 0]])
Notebook on Google Colab (where the same code yields yet a different error, strangely):

How to encode integer masks as bits into dummy variables in pandas
I would like to encode integer masks stored in pandas dataframe column into respective binary features which correspond to bit positions in those integers. For example, given 4bit integers, and a decimal value of 11 I would like to derive 4 columns with values 1, 0, 1, 1, and so on across entire column.

Converting images to onehot array and back returns black images
I have a numpy array of numpy array of 3 grayscale images with just 0 and 255 values (
shape: (3, 512, 512)
). I turned them to onehot encodings into 2 classes usingtf.one_hot
(shape: (3, 512, 512, 2)
). Now, I am trying to get them back to original form throughnp.argmax
. But when I read the images, they are all black.labels = [] labels.append(label_1) labels.append(label_2) labels.append(label_3) labels = np.asarray(labels) print(labels.shape) # (3, 512, 512) sess = tf.InteractiveSession() labels = tf.one_hot(labels, 2) # (3, 512, 52, 2) print(labels.shape) #print(labels) labels = labels.eval() # to convert to numpy array from tensor imgs = np.argmax(labels, axis=3) # also tried tf.argmax print(imgs.shape) # (3, 512, 512) for i in range(imgs.shape[0]): image = imgs[i] print(img.shape) # (512, 512) indices = np.where(image > 0) print(indices) # array([], dtype=int64), array([], dtype=int64) print(indices) image = Image.fromarray(image, 'L') image.show() # black images, values all zero
I am sure I am missing something pretty simple but can't figure it out. Any help will be appreciated. Thanks.
Edit:
I checked for indices where the array has nonzero values. But all values seem to be zeros. So, not an issue of displaying the image. I think the problem is with
argmax
but I do not know what.