How to utilize user survey answers and the actual usage in forecasting power usage using LSTM?
I have the pretrial survey and posttrial survey conducted of around 5000 users for Smart Meter installation.
With this I have power usage reading recorded every 30 min for around one and a half years.
survey csv is like this
User Question 1 Question 2 .......
1000 a a .......
1001 b a .......
. . . .......
. . . .......
. . . .......
5000 b a .......
power usage csv is like this
User date usage
1000 20001 0.003
1000 20002 0.025
.... ..... .....
.... ..... .....
.... ..... .....
.... ..... .....
.... ..... .....
1000 65047 0.52
1000 65048 0.14
I want to forecast power usage of the user based on the past power usage and the survey using LSTM. How to start with this?
See also questions close to this topic

How to pass epoch and batch size when using label powerset in keras
I have a multilabel problem and with some research, I was able to use Label powerset in conjunction with ML algorithms. Now I want to use the Label powerset with neural network and as per the official website I can use Label powerset. But I am not able to understand how to modify my existing code to be able to use Label Powerset.
I want to know how can we pass epoch or batch_size or any other parameter passed in the fit function of the model.
Since I have a multilabel problem I have used MultiLabelBinarizer of sklearn so my each target row looks like this [1,0,0,1,0,0,0,0,0,0,0,0].
and lastly, if someone could explain to me what is KERAS_PARAMS and Keras() in the below line:
def create_model_multiclass(input_dim, output_dim): # create model model = Sequential() model.add(Dense(8, input_dim=input_dim, activation='relu')) model.add(Dense(output_dim, activation='softmax')) # Compile model model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) return model clf = LabelPowerset(classifier=Keras(create_model_multiclass, True, KERAS_PARAMS), require_dense=[True,True]) clf.fit(X_train,y_train) y_pred = clf.predict(X_test)
Below is my existing neural network code
cnn_model = Sequential() cnn_model.add(Dropout(0.5)) cnn_model.add(Conv1D(25,7,activation='relu')) cnn_model.add(MaxPool1D(2)) cnn_model.add(Dropout(0.2)) cnn_model.add(Conv1D(25,7,activation='relu')) cnn_model.add(MaxPool1D(2)) cnn_model.add(Flatten()) cnn_model.add(Dense(25,activation='relu')) cnn_model.add(Dense(12,activation='softmax')) cnn_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc']) history = cnn_model.fit(X_train, y_train, validation_data=(X_test,y_test), batch_size=32, epochs=180,verbose=1) plot_history(history) predictions = cnn_model.predict(X_test)
I want my output row to look like this only [1,0,0,1,0,0,0,0,0,0,0,0] as later I will use my MultiLabelBinarizer for the inverse transform of this.

Loss functions reaching global minima
In Deep Learning can we have train accuracy far less than 100% at the global minimum of the loss function?
I have coded a neural network in python to classify cats and noncats. I chose a 2layer network. It gave a train accuracy of 100% and a test accuracy of 70%.
When I increased the #layers to 4 the loss function is getting stuck at 0.6440 leading to train accuracy of 65% and a test accuracy of 34% for many random initializations.
We are expecting that our train accuracy on the 4layer model should be 100%. But we are getting stuck at 65%. We are thinking that the loss function is reaching a global minimum since on many random initialization we are stagnating at a loss value of 0.6440. So, even though the loss function is reaching the global minimum, why is the train accuracy not reaching 100%? Hence our question,"In Deep Learning can we have train accuracy nonzero at the global minimum of the loss function?"

Should I use activation function and normalization for regression?
I have a regression problem, a model in related paper use a minmax normalization to normalize input data and output data to 1 to 1 range, and it apply a tanh activation function in the last output layer. However, I found out it's very hard to train, the loss and rmse decrease slowly. If I remove the activation function in output layer, and do not use any data normalization, it get the best score. So, I have two questions:
Do I have to use some activation function in last layer and some data normalization for a regression problem? (all features and true value are in the same scale, just like the house price in different area etc..)
Even I remove the activation function in last layer, but I found out that if I don't use any data normalization, the loss decrease more faster. If I normalize the data to 1 to 1 range or 0 to 1 range (use minmax normalization), the result is always worse. But, why?

Using RNN and LSTM for process control using available data mesaurements
I have 50 sample measurements of input (temperature) with 600 time steps in each sample. I also have the outputs for these 50 samples in form of 4 variables (like saturation, concentration, lower bound concentration and upper bound concentration).
So the input data is 50 samples, 600 timesteps each and 1 variable or property. The output is 50 samples, 600 timesteps each and 4 variables or properties (and the variable 2 concentration must always be between the variable 3 and 4 which are its lower and upper bounds at all times)
I am new to deep learning and LSTMs. I am having trouble implementing the LSTM. So from what I have read (kindly correct me if I am wrong), I need to use many to many LSTM and batch_size = 50 input_shape is (600,) seq_len = 1
How many units of LSTM to use? I am very confused how to implement LSTM in this.

Extremely poor accuracy and high training time per epoch for a word level language model
Setting up my first wordlevel language model using the Keras API. Extremely poor accuracy results and unbelievably high training time required.
I developed my first wordlevel language model using the Keras library with my training set as the script for Pulp Fiction. I cleaned the text of all punctuation and converted all the words to lower case. When I start training my model on the given dataset it starts off with an accuracy of 3% and a training time of 67 mins per epoch. This is extremely demotivating and I was wondering whether I should tune my hyperparameters or this is normal behaviour for my model and it will yield better results with more epochs?
model=Sequential() model.add(LSTM(256, input_shape=(X.shape[1], X.shape[2]), return_sequences=True)) model.add(Dropout(0.5)) model.add(LSTM(256)) model.add(Dropout(0.5)) model.add(Dense(y.shape[1], activation='softmax')) model.compile(optimizer='adam',loss='categorical_crossentropy', metrics= ['accuracy']) model.fit(X,y,batch_size=128,epochs=100)

how to reshape text data to be suitable for LSTM model in keras
My shape of data is
(87716, 200)
and I want to reshape it in a way I can feed it into LSTM.I have a code for
LSTM Autoencoder
. and the below is the architecture of my mode:inputs = Input(shape=(SEQUENCE_LEN,VOCAB_SIZE), name="input") # inputs = Embedding( VOCAB_SIZE, 256, input_length=SEQUENCE_LEN)(input) encoded = Bidirectional(LSTM(LATENT_SIZE), merge_mode="sum", name="encoder_lstm")(inputs) decoded = RepeatVector(SEQUENCE_LEN, name="repeater")(encoded) decoded = LSTM(VOCAB_SIZE, return_sequences=True)(decoded) autoencoder = Model(inputs, decoded) autoencoder.compile(optimizer="sgd", loss='mse') autoencoder.summary() history = autoencoder.fit(Xtrain, Xtrain,batch_size=BATCH_SIZE, epochs=NUM_EPOCHS)
And this is the way I have prepared the data to feed to model:
BATCH_SIZE=512 VOCAB_SIZE = 4000 SEQUENCE_LEN = 200 sent_wids = np.zeros((len(parsed_sentences),SEQUENCE_LEN),'int32') sample_seq_weights = np.zeros((len(parsed_sentences),SEQUENCE_LEN),'float') for index_sentence in range(len(parsed_sentences)): temp_sentence = parsed_sentences[index_sentence] temp_words = nltk.word_tokenize(temp_sentence) for index_word in range(SEQUENCE_LEN): if index_word < sent_lens[index_sentence]: sent_wids[index_sentence,index_word] = lookup_word2id(temp_words[index_word]) else: sent_wids[index_sentence, index_word] = lookup_word2id('PAD') print(sent_wids.shape) #(87716, 200) Xtrain = sent_wids
Now how can I reshape the data in which I can feed it to this model: I am getting this error :
ValueError: Error when checking input: expected input to have 3 dimensions, but got array with shape (87716, 200)
When I do reshaping like this
Xtrain = Xtrain.reshape(1,SEQUENCE_LEN, VOCAB_SIZE)
.What do we need to do if our prepared data is not dividable by the shapes and batch_size we considered?
Please let me know which part is not clear I will explain.
Thanks for your help:)

Unable to get the forecast_ets property of the worksheetfunction class
I've been trying to call the
Forecast.ETS
function from VBA code withApplication.WorksheetFunction
, however I'm getting the "unable to get theforecast_ets
property of the worksheetfunction class error."Sub test() Cells(20, 5) = Application.WorksheetFunction.Forecast_ETS(Cells(5, 3).Value, _ Range(Cells(5, 3), Cells(20, 3)), _ Range(Cells(5, 2), Cells(20, 2)).Value) End Sub
Cells(5,3).Value
is a date, same asRange(Cells(5,2),Cells(20,2)).Value
Range(Cells(5, 3), Cells(20, 3))
contains the historical values. 
Why does RMSE increase with horizon when using the timeslice method in caret's trainControl function?
I'm using the
timeslice
method in caret'strainControl
function to perform crossvalidation on a time series model. I've noticed that RMSE increases with thehorizon
argument.I realise this might happen for several reasons, e.g., if explanatory variables are being forecast and/or there's autocorrelation in the data such that the model can better predict nearer vs. farther ahead observations. However, I'm seeing the same behaviour even when neither is the case (see trivial reproducible example below).
Can anyone explain why RSMEs are increasing with
horizon
?# Make data X = data.frame(matrix(rnorm(1000 * 3), ncol = 3)) X$y = rowSums(X) + rnorm(nrow(X)) # Iterate over different different forecast horizons and record RMSES library(caret) forecast_horizons = c(1, 3, 10, 50, 100) rmses = numeric(length(forecast_horizons)) for (i in 1:length(forecast_horizons)) { ctrl = trainControl(method = 'timeslice', initialWindow = 500, horizon = forecast_horizons[i], fixedWindow = T) rmses[i] = train(y ~ ., data = X, method = 'lm', trControl = ctrl)$results$RMSE } print(rmses) #0.7859786 0.9132649 0.9720110 0.9837384 0.9849005

Multivariate time series forecasting lstm
Assume I have 200 samples of time series data. The input has three features and I want to forecast one step ahead in time. So assume my series look like the following: The first three rows:
1 2 3 4 5 6 7 8 9
Now I will convert the problem to supervised learning problem for one step prediction:
Input Output 1 2 3 4 5 6 4 5 6 7 8 9
Now I would like to forecast the value of all three features in the output not only one of them. Now I would like to know what should be the dimension of the input to Keras?: I think it should be
(100,1,3)
where 100 can be replaced by the batch size. Can someone verify this. The main question is what should I change in lstm configuration for it to understand that I need multivariate setting for the output like for example should I havedense(3)
to specify I need 3 outputs. I would be appreciate if someone can help.