How to utilize user survey answers and the actual usage in forecasting power usage using LSTM?
I have the pre-trial survey and post-trial survey conducted of around 5000 users for Smart Meter installation.
With this I have power usage reading recorded every 30 min for around one and a half years.
survey csv is like this
User Question 1 Question 2 ....... 1000 a a ....... 1001 b a ....... . . . ....... . . . ....... . . . ....... 5000 b a .......
power usage csv is like this
User date usage 1000 20001 0.003 1000 20002 0.025 .... ..... ..... .... ..... ..... .... ..... ..... .... ..... ..... .... ..... ..... 1000 65047 0.52 1000 65048 0.14
I want to forecast power usage of the user based on the past power usage and the survey using LSTM. How to start with this?
See also questions close to this topic
How to penalise my agent for using different action?
I'm using A3C in tensroflow, I want to keep my agent using the same action and only change when necessary and will have a benefit from changing.
what functions of OPENCV mainly used in image analysis or particularly in deep learning?
I want to know about what functions or how we use OPENCV in analysis the image which is overall beneficial for deep learning models?
- How to implement asymmetric structure of LeNet-5 in tensorflow?
LSTM Training Input Versus Live Evaluation Input - Dynamic RNN?
I am having trouble wrapping my head around RNNs for this problem.
The problem: live binary classification of video using image sequences. Meaning I am receiving a video one image at a time and need to predict either Class A or Class B for the most recent image received.
Training - I use a CNN as feature extractor on a full sequence of images. I then feed multiple images (lstm-len, cnn-feature-size) into the LSTM.
Live Evaluation - I receive 1 frame at a time and run it through the CNN. I add these new features to a queue of length lstm-len, then I take all the features from the queue and feed into the LSTM.
What I don't understand
Why is it that I have to keep track of and feed all of the features into the LSTM at evaluation time? The point of an LSTM is to remember past inputs so it seems redundant for me to input all the previous images at every time step. What I would like to be able to do is simply calculate the features for the most recent images and then feed those new features into the LSTM while the LSTM remembers the last lstm-len number of frames.
Am I using the RNN incorrectly in this case? Should I be able to simply use the previous LSTM state as input into the other LSTM cells and provide feature input for only the newest image?
I'm thinking something like tensorflows DynamicRNN may be the solution to this problem.
Pretty confused about this. Thanks for the help!
Reinforcement learning - How to deal with varying number of actions which do number approximation
I am a new to Reinforcement learning, but I am trying to use RL in this task:
Given a function definition in written e.g. in C with 1 to 10s of input arguments (only numerical ones - integer, float, etc.) and the body of the function (represented as a Abstract Syntax Tree/ Abstract Decision Tree with data dependencies - how the internal variable values change) I would like to approximate the values of these input parameters so for e.g. a certain decision block is executed. For this I thought of a recurrent network with LSTM cells.
Now, to achieve this, I would traverse one path in the tree leading to the block and take note of any data changes and decision blocks in the path. These steps would influence my parameter input predictions - what values to insert into/change in the input parameters if I wish to have a certain decision block executed.
Action: Changing the value of one chosen input parameter of the function OR Changing the value of all input parameters individually (with mathematical different operation). After action execution, moving onto the next node in the tree.
Reward: How close I am to executing the given decision block (thus satisfying the condition) with given input parameter values.
Goal: Have a condition in the code satisfied and a decision block thus executed (e.g. an if condition is met).
State: Current position in the AST/ADT with data dependencies.
Assuming that I already have a way to evaluate, how far I am from executing the wanted decision block given current parameter input values, I came across two problems:
How would I deal with varying number of function input parameters in RL? If I want to change their values to be closer to the execution of the wanted decision block, the number of given actions changes with the number of parameters defined for the given function.
If I already did chose one parameter, what is the best way to do number approximation using RL? In the function body there could be numerous very complex mathematical operations happening, so should there be defined action as logarithm, exponentiation, division, multiplying, etc. or is there a better way with maybe just adding/subtracting from the current value?
If you find any mistakes in my definition of the Actions, Reward, Goal or State, please do correct me, as I am still a big learner in this field.
Thank you for your answers.
LSTM Neural Network weighted connections
I'm currently implementing my own LSTM neural network using the neat algorithm. Everything about LSTM seems pretty clear so far besides the weights: Are the weights unique per lstm-cell or do they depend on their output connection?
For example, if one cell has outgoing connections does every connection hold different weights for all input/reccurrent datas (Just like in a simple non RNN network where every connection has a weight assigned) of that cell or are they unique for all cells (same output over every outgoing connection)?
Turning a sales data set into a hierarchical time series in R
I have a sales time series that is sales information for different styles each day and am having trouble converting it into a hierarchical time series for use with the hts package in R
The columns of the data are: date, department, brand, style, sales units
Does anyone know of a way to mutate a data and convert it into a bottom level time series that can be put into hts? And also a way to get number of nodes without having to count each department, brand, etc
Error while calculating accuracy of a timeseries model in R (Error in NextMethod(.Generic) : (list) object cannot be coerced to type 'double')
With this code:
#plotting time series from year 1998 to 2008 year.time_series <- ts(t_AMOUNT,start = c(1998) , frequency = 12 ) #Monthly 12 plot(year.time_series) #splitting the timeseries for further model evaluation train <- window(year.timeseries, start=1998,end=2005) test <- window(year.timeseries, start=2005, end=2008) #using models to check the accuracy results etsfit <- ets(train) summary(etsfit) plot(train, main="ETS Forecast", ylab = "ets(training set)", cex.lab = 1.5, cex.main = 1.5, cex.axis = 1.5) lines(etsfit$fitted, col="orange") #forecast forecast.ets <- forecast(etsfit, h=24) summary(forecast.ets) plot(forecast.ets) plot(forecast.ets, main = "2 Year Forecast Using ETS Model", xlim = c(1998, 2008), cex.lab = 1.5, cex.main = 1.5, cex.axis = 1.5) lines(test, col="red") library(Metrics) #input = forecast values, actual values accuracy(forecast.ets,test)
I am getting the following error at
Error in NextMethod(.Generic) :
(list) object cannot be coerced to type 'double'
In addition: Warning message:
!=.default(actual, predicted) :
longer object length is not a multiple of shorter object length
Is there a way to split a time series and calculate it's accuracy?
Excel planned vs actual stopping actual line after todays date
I have created a Excel Line graph comparing cummalitive Planned vs Forecast vs actual dates for testing of equipment. I have managed to create the graph however I would like to use a formula that will stop the "actual" cummalitive line after todays date (instead of just carrying on as it is currently doing see image) Can anyone help?
I have attempted =IF(data actual>TODAY(), NA(), COUNTIF(Data actual, date in top row column))