Should I use teacher forcing for chatbot?
I'm a little confused regarding the implementation of teacher forcing(Keras&Tensforflow) and don't know whether it should be used for chatbot training.
See also questions close to this topic

Shape Error in Andrew NG Logistic Regression using Scipy.opt
I've been trying to write Andrew NG's Logistic Regression Problem Using python and Scipy.opt for optimizing the function. However, I get a VALUE ERROR that says I have mismatching dimensions. I've tried to flatten() my theta array as scipy.opt doesn't seem to work very well with single column/row vector, however the problem still persists.
Kindly point me in the right direction as to what is causing the problem and how to avoid it.
Thanks a million!
import pandas as pd import numpy as np import matplotlib.pyplot as plt import scipy.optimize as opt dataset = pd.read_csv("Students Exam Dataset.txt", names=["Exam 1", "Exam 2", "Admitted"]) print(dataset.head()) positive = dataset[dataset["Admitted"] == 1] negative = dataset[dataset["Admitted"] == 0] #Visualizing Dataset plt.scatter(positive["Exam 1"], positive["Exam 2"], color="blue", marker="o", label="Admitted") plt.scatter(negative["Exam 1"], negative["Exam 2"], color="red", marker="x", label="Not Admitted") plt.xlabel("Exam 1 Score") plt.ylabel("Exam 2 Score") plt.title("Admission Graph") plt.legend() #plt.show() #Preprocessing Data dataset.insert(0, "x0", 1) col = len(dataset.columns) x = dataset.iloc[:,0:col1].values y = dataset.iloc[:,col1:col].values b = np.zeros([1,col1]) m = len(y) print(f"X Shape: {x.shape} Y Shape: {y.shape} B Shape: {b.shape}") #Defining Functions def hypothesis(x, y, b): h = 1 / (1+np.exp(x @ b.T)) return h def cost(x, y, b): first = (y.T @ np.log(hypothesis(x, y, b))) second = (1y).T @ np.log(1  hypothesis(x, y, b)) j = (1/m) * np.sum(first+second) return j def gradient(x, y, b): grad_step = ((hypothesis(x, y, b)  y) @ x.T) / m return b #Output initial_cost = cost(x, y, b) print(f"\nInitial Cost = {initial_cost}") final_cost = opt.fmin_tnc(func=cost, x0=b.flatten() , fprime=gradient, args=(x,y)) print(f"Final Cost = {final_cost} \nTheta = {b}")
Dataset Used: ex2.txt
34.62365962451697,78.0246928153624,0 30.28671076822607,43.89499752400101,0 35.84740876993872,72.90219802708364,0 60.18259938620976,86.30855209546826,1 79.0327360507101,75.3443764369103,1 45.08327747668339,56.3163717815305,0 61.10666453684766,96.51142588489624,1 75.02474556738889,46.55401354116538,1 76.09878670226257,87.42056971926803,1 84.43281996120035,43.53339331072109,1 95.86155507093572,38.22527805795094,0 75.01365838958247,30.60326323428011,0 82.30705337399482,76.48196330235604,1 69.36458875970939,97.71869196188608,1 39.53833914367223,76.03681085115882,0 53.9710521485623,89.20735013750205,1 69.07014406283025,52.74046973016765,1 67.94685547711617,46.67857410673128,0 70.66150955499435,92.92713789364831,1 76.97878372747498,47.57596364975532,1 67.37202754570876,42.83843832029179,0 89.67677575072079,65.79936592745237,1 50.534788289883,48.85581152764205,0 34.21206097786789,44.20952859866288,0 77.9240914545704,68.9723599933059,1 62.27101367004632,69.95445795447587,1 80.1901807509566,44.82162893218353,1 93.114388797442,38.80067033713209,0 61.83020602312595,50.25610789244621,0 38.78580379679423,64.99568095539578,0 61.379289447425,72.80788731317097,1 85.40451939411645,57.05198397627122,1 52.10797973193984,63.12762376881715,0 52.04540476831827,69.43286012045222,1 40.23689373545111,71.16774802184875,0 54.63510555424817,52.21388588061123,0 33.91550010906887,98.86943574220611,0 64.17698887494485,80.90806058670817,1 74.78925295941542,41.57341522824434,0 34.1836400264419,75.2377203360134,0 83.90239366249155,56.30804621605327,1 51.54772026906181,46.85629026349976,0 94.44336776917852,65.56892160559052,1 82.36875375713919,40.61825515970618,0 51.04775177128865,45.82270145776001,0 62.22267576120188,52.06099194836679,0 77.19303492601364,70.45820000180959,1 97.77159928000232,86.7278223300282,1 62.07306379667647,96.76882412413983,1 91.56497449807442,88.69629254546599,1 79.94481794066932,74.16311935043758,1 99.2725269292572,60.99903099844988,1 90.54671411399852,43.39060180650027,1 34.52451385320009,60.39634245837173,0 50.2864961189907,49.80453881323059,0 49.58667721632031,59.80895099453265,0 97.64563396007767,68.86157272420604,1 32.57720016809309,95.59854761387875,0 74.24869136721598,69.82457122657193,1 71.79646205863379,78.45356224515052,1 75.3956114656803,85.75993667331619,1 35.28611281526193,47.02051394723416,0 56.25381749711624,39.26147251058019,0 30.05882244669796,49.59297386723685,0 44.66826172480893,66.45008614558913,0 66.56089447242954,41.09209807936973,0 40.45755098375164,97.53518548909936,1 49.07256321908844,51.88321182073966,0 80.27957401466998,92.11606081344084,1 66.74671856944039,60.99139402740988,1 32.72283304060323,43.30717306430063,0 64.0393204150601,78.03168802018232,1 72.34649422579923,96.22759296761404,1 60.45788573918959,73.09499809758037,1 58.84095621726802,75.85844831279042,1 99.82785779692128,72.36925193383885,1 47.26426910848174,88.47586499559782,1 50.45815980285988,75.80985952982456,1 60.45555629271532,42.50840943572217,0 82.22666157785568,42.71987853716458,0 88.9138964166533,69.80378889835472,1 94.83450672430196,45.69430680250754,1 67.31925746917527,66.58935317747915,1 57.23870631569862,59.51428198012956,1 80.36675600171273,90.96014789746954,1 68.46852178591112,85.59430710452014,1 42.0754545384731,78.84478600148043,0 75.47770200533905,90.42453899753964,1 78.63542434898018,96.64742716885644,1 52.34800398794107,60.76950525602592,0 94.09433112516793,77.15910509073893,1 90.44855097096364,87.50879176484702,1 55.48216114069585,35.57070347228866,0 74.49269241843041,84.84513684930135,1 89.84580670720979,45.35828361091658,1 83.48916274498238,48.38028579728175,1 42.2617008099817,87.10385094025457,1 99.31500880510394,68.77540947206617,1 55.34001756003703,64.9319380069486,1 74.77589300092767,89.52981289513276,1

Are there pretrained model to recognize character?
I extracted from numberplate into individual characters as picture.
https://i.stack.imgur.com/k09UB.jpg
I wonder there are pretrained CNN model to this recognize characters. If any, please recommend for me API to perform this. Thanks a lot.

How to tune hyper parameters of LightGBM using bayesian optimization with rmse as metric
I'm trying to tune hyperparameters of lightgbm model with rmse as metric. lgbBO = BayesianOptimization(lgb_eval, {'num_leaves': (24, 45), 'feature_fraction': (0.1, 0.9), 'bagging_fraction': (0.8, 1), 'max_depth': (5, 8.99), 'lambda_l1': (0, 5), 'lambda_l2': (0, 3), 'min_split_gain': (0.001, 0.1), 'min_child_weight': (5, 50)}, random_state=0) after this what should i do?

Split the text in paragraphs
I know that I can use something like this
theText='She loves music. Her favorit instrument is the piano.\n\n However, \n\n she does not play it.' paragraphs = [p for p in theText.split('\n\n') if p] for i,p in enumerate(paragraphs): print(i,p)
to split theText in paragraphs.
However, I would like to add an additional conditional that the next sentence cannot start with a lower case letter. The actual code provides
0 She loves music. Her favorit instrument is the piano. 1 However, 2 she does not play it.
I would like
0 She loves music. Her favorit instrument is the piano. 1 However, she does not play it.
I believe that I should use some regex, but I could not figure out the correct structure.

Calculating edit distance on successive rows of a `Spark Dataframe
I have a data frame as follows:
import org.apache.spark.sql.expressions.Window import org.apache.spark.sql.Column import org.apache.spark.sql.functions._ import spark.implicits._ // some data... val df = Seq( (1, "AA", "BB", ("AA", "BB")), (2, "AA", "BB", ("AA", "BB")), (3, "AB", "BB", ("AB", "BB")) ).toDF("id","name", "surname", "array") df.show()
and i am looking to calculate the edit distance between the 'array' column in successive row. As an example i want to calculate the edit distance between the 'array' entity in column 1 ("AA", "BB") and the the 'array' entity in column 2 ("AA", "BB"). Here is the edit distance function i am using:
def editDist2[A](a: Iterable[A], b: Iterable[A]): Int = { val startRow = (0 to b.size).toList a.foldLeft(startRow) { (prevRow, aElem) => (prevRow.zip(prevRow.tail).zip(b)).scanLeft(prevRow.head + 1) { case (left, ((diag, up), bElem)) => { val aGapScore = up + 1 val bGapScore = left + 1 val matchScore = diag + (if (aElem == bElem) 0 else 1) List(aGapScore, bGapScore, matchScore).min } } }.last }
I know i need to create a UDF for this function but can't seem to be able to. If i use the function as is and using Spark Windowing to get at the pervious row:
// creating window  ordered by ID val window = Window.orderBy("id") // using the window with lag function to compare to previous value in each column df.withColumn("editd", editDist2(($"array"), lag("array", 1).over(window))).show()
i get the following error:
<console>:245: error: type mismatch; found : org.apache.spark.sql.ColumnName required: Iterable[?] df.withColumn("editd", editDist2(($"array"), lag("array", 1).over(window))).show()

Split the cells, create columns, count
It's a multistep exercise that, I suspect, can be handled in various ways. Here is what I have and have done.
tableA contains Stores and Brands. tableB contains Customer and Stores. Tables can be joined on Stores.
tableA = [(Ikea, 'Adidas, Nike'), (Target, 'Adidas, NB'), (Sears, 'Puma')] labels = ['Store', 'Brand'] dfA = pd.DataFrame.from_records(tableA, columns=labels) tableB = [('Neil', Ikea), ('Neil', Target), ('Javal', Target), ('Colleen', Ikea), ('Colleen', Sears), ('Javal', Target), ('Neil', Target), ('Colleen', Sears)] labels = ['Customer', 'Store'] dfB = pd.DataFrame.from_records(tableB, columns=labels)
Customers as rows, brands as columns and count as values.
First, I want to deal with splitting the cells and counting. Later, I will join two tables.
Splitting
The best I am able to achieve is:
dfA['Adidas'], dfA['Nike'] = dfA['tags'].str.split(', ').str
If I do:
dfA['Adidas'], dfA['Nike'], dfA['NB'], dfA['Puma'] = dfA['tags'].str.split(', ').str
I get a mistake:
ValueError: not enough values to unpack (expected 4, got 2)
I understand the mistake's nature but haven't found an alternative yet.
Questions I have:
(1) Should I first deal with splitting and then join tables?
(2) How to properly split the column?
(3) How to add proper counts (Counter has nothing to do with it, right?)

Botpress: Custom Content Type and Custom Rendering
I am new to Botpress.
I just installed the latest version ‘botpresscev11_0_1winx64’ of Botpress. I was going over the documentation and i found some explanation about Content Type, Content Element and Content Rendering. But it is not very clear as to how I can create a custom type and it’s rendering.
Also reading further in the section ‘Creating Modules’ it says: “and soon they will be able to add new content types and content elements”.
so, I am not able to clearly understand how can i create custom content type and it’s rendering that is specific to my domain.
I am trying to understand if I have to add custom elements like Radio Buttons or Checkboxes or Dropdowns in my response that is displayed in the web chat for instance, how can i achieve that.
I would highly appreciate if someone can provide some example or link to the documentation.
Thanks in advance.

Recast: reset all memory except specific field
Is there a way, while configuring recast bot, to set an action to reset all memory, except a specific field?
I already tried to unset all fields except the one I need, but it's boring, unextendable, unscalable and unmaintenable.
Thanks

Google Assistant ends conversation
We're currently testing our chatbot project for Google Assistant/Google Home.
In the middle of the conversation, when our bot still has "things to say" (our chatbot flows are based on several steps and not just one question and one short answer), Google Assistant throws you out of the chatbot and goes back to the generic Google Assistant. Does anyone know if this is a feature of the testing version or if this is caused by the way Google Assistant works?

Can an RNN/LSTM have a different rank input than its output
I'm trying to implement an LSTM that takes the SWT of a time series as an input to predict the value of the time series in the near future steps.
The input is dimension M X t, where M > 1 The output is dimension t .
is it possible to have a higher rank input to a network in tensor flow or am I required to concatenate my input to match the output dimension?

Getting multidimensional predictions with Tensorflow session.run() and rnn.static_rnn()
After many hours stuck on that problem and searching for solutions, I come here for help. I am a beginner with Tensorflow.
I have setup an LSTM prediction algorithm in Python using Tensorflow, based on this and this. Unfortunately when I run the code, I get a multidimensional prediction. Which generates the error:
> symbols_out_pred = analysed_reversed_dictionary[int(tf.argmax(onehot_pred, 1).eval())]
TypeError: only size1 arrays can be converted to Python scalars
The prediction (variable onehot_pred) has shape (3, 7), which is (batch_size, vocab_size) instead of being a one dimensional array of 7 values (vocab_size). What does it mean? And how can I have a onedimensional prediction?
Here is the simplified code with comments:
# Parameters learning_rate = 0.001 training_iters = 50 display_step = 10 time_steps = len(feature_columns) batch_size = 3 analysed_column = 0 # number of units in RNN cell n_hidden = 512 # tf Graph input vocab_size = len(dictionaries[feature_columns[analysed_column]]) x = tf.placeholder("float", [None, time_steps, 1]) y = tf.placeholder("float", [None, vocab_size]) # RNN output node weights and biases weights = { 'out': tf.Variable(tf.random_normal([n_hidden, vocab_size])) } biases = { 'out': tf.Variable(tf.random_normal([vocab_size])) } # Target log path logs_path = '/tmp/tensorflow/rnn_words' writer = tf.summary.FileWriter(logs_path) def RNN(x, weights, biases): #processing the input tensor from [batch_size, time_steps, 1] to "time_steps" number of [batch_size, 1] tensors x=tf.unstack(x ,time_steps,1) # 2layer LSTM, each layer has n_hidden units. rnn_cell = rnn.MultiRNNCell([tf.nn.rnn_cell.LSTMCell(n_hidden),tf.nn.rnn_cell.LSTMCell(n_hidden)]) # generate prediction outputs, states = rnn.static_rnn(rnn_cell, x, dtype=tf.float32) # there are n_input outputs but # we only want the last output return tf.matmul(outputs[1], weights['out']) + biases['out'] pred = RNN(x, weights, biases) # Loss and optimizer cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=pred, labels=y)) optimizer = tf.train.RMSPropOptimizer(learning_rate=learning_rate).minimize(cost) # Model evaluation correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1)) accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) # Initializing the variables init = tf.global_variables_initializer() # Launch the graph start_time = time() with tf.Session() as session: session.run(init) step = 0 offset = randint(0,batch_size+1) end_offset = batch_size + 1 acc_total = 0 loss_total = 0 writer.add_graph(session.graph) analysed_dictionary = dictionaries[feature_columns[analysed_column]] analysed_reversed_dictionary = reverse_dictionaries[feature_columns[analysed_column]] while step < training_iters: # Generate a minibatch. Add some randomness on selection process (once all set browsed). if offset > (len(training_data)end_offset): offset = randint(0, batch_size+1) symbols_in_keys = [] for i in range(offset, offset+batch_size): symbols_in_keys.append( [dictionaries[feature][training_data[i][feature_i]] for feature_i, feature in enumerate(feature_columns)] ) symbols_in_keys = np.reshape(np.array(symbols_in_keys), [batch_size,time_steps,1]) symbols_out_onehot = np.zeros([vocab_size], dtype=float) correct_pred = training_data[offset+batch_size][analysed_column] symbols_out_onehot[analysed_dictionary[correct_pred]] = 1.0 symbols_out_onehot = np.reshape(symbols_out_onehot,[1,1]) _, acc, loss, onehot_pred = session.run([optimizer, accuracy, cost, pred], \ feed_dict={x: symbols_in_keys, y: symbols_out_onehot}) loss_total += loss acc_total += acc if (step+1) % display_step == 0: print("Iter= " + str(step+1) + ", Average Loss= " + \ "{:.6f}".format(loss_total/display_step) + ", Average Accuracy= " + \ "{:.2f}%".format(100*acc_total/display_step)) acc_total = 0 loss_total = 0 symbols_in = [training_data[i] for i in range(offset, offset + batch_size)] symbols_out = training_data[offset + batch_size] symbols_out_pred = analysed_reversed_dictionary[int(tf.argmax(onehot_pred, 1).eval())] print("%s  [%s] vs [%s]" % (symbols_in,symbols_out,symbols_out_pred)) step += 1 offset += (batch_size+1)
The line generating the variable is
_, acc, loss, onehot_pred = session.run([optimizer, accuracy, cost, pred], \ feed_dict={x: symbols_in_keys, y: symbols_out_onehot})
and the function incriminated is RNN.
Thank you a lot for your answers!

ValueError: Error when checking target: expected dense_3 to have 2 dimensions, but got array with shape (500, 10, 14)
Keras: 2.1.6, python 3.6, tensorflow 1.8.0
I am trying to train a Sequence Model that has two LSTM layers and 3 dense layers. I had done some data preparation beforehand and setup my data in the format the LSTM layer requires it, i.e.
(n_samples, n_timesteps, n_features)
. My data has 14 features and is actually a long sequence of 5000 steps thus I have broken it down to 500 samples to 10 time steps each. Once I completed that, I started with below model, but soon ran into the error of Input shape for the last layer. I tried using Sequential and Functional API both yield the same error.import keras from keras import callbacks import tensorflow as tf from keras.models import Model from keras.layers import Input, Dense from keras.layers import LSTM X_input = Input(X_train.shape[1:]); ## First LSTM Layer X = LSTM(10, return_sequences=True, input_shape = (10,14), name = 'LSTM_1')(X_input); ## Second LSTM Layer X = LSTM(10)(X); ## First Dense Layer X = Dense(10, activation='relu', name = 'dense_1')(X) ## Second Dense Layer X = Dense(5, activation='relu', name = 'dense_2')(X) ## Final Dense Layer X = Dense(1, activation='relu', name = 'dense_3')(X) ##The model object model = Model(inputs = X_input, outputs = X, name='LSTMModel') model.compile(optimizer = "Adam" , loss = "mean_squared_error", metrics = ['mean_squared_error','cosine', 'mae']); Model.fit(x = X_train, y = Y_train, epochs = 300, callbacks=[tensorboard], validation_data=(X_eval,Y_eval));
My data is of shape
(500,10,14)
:>>> X_train.shape (500,10,14)
And my model summary looks like such:
model.summary() _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) (None, 10, 14) 0 _________________________________________________________________ LSTM_1 (LSTM) (None, 10, 10) 1000 _________________________________________________________________ LSTM_2 (LSTM) (None, 10) 840 _________________________________________________________________ dense_1 (Dense) (None, 10) 110 _________________________________________________________________ dense_2 (Dense) (None, 5) 55 _________________________________________________________________ dense_3 (Dense) (None, 1) 6 ================================================================= Total params: 2,011 Trainable params: 2,011 Nontrainable params: 0 _________________________________________________________________
Although, I still get the error:
ValueError: Error when checking target: expected dense_3 to have 2 dimensions, but got array with shape (500, 10, 14)
The shape of my labels are as follows:
X_train = np.reshape(Train_data_scaled.values,(500,10,14)); Y_train = np.reshape(Train_labels_scaled.values,(500,10,1)); X_eval = np.reshape(Validation_data_scaled.values,(10,10,14)); Y_eval = np.reshape(Validation_labels_scaled.values,(10,10,1));
After adding the layer of RepeatVector I find another issue here is the stack trace of the same.
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) (None, 10, 14) 0 _________________________________________________________________ LSTM_1 (LSTM) (None, 10) 1000 _________________________________________________________________ repeat_vector_1 (RepeatVecto (None, 10, 10) 0 _________________________________________________________________ LSTM_2 (LSTM) (None, 10, 10) 840 _________________________________________________________________ dense_1 (Dense) (None, 10, 10) 110 _________________________________________________________________ dense_2 (Dense) (None, 10, 5) 55 _________________________________________________________________ dense_3 (Dense) (None, 10, 1) 6 ================================================================= Total params: 2,011 Trainable params: 2,011 Nontrainable params: 0 _________________________________________________________________ Traceback (most recent call last): File ".\lstm.py", line 76, in <module> tf.app.run() File "C:\Program Files\Python36\lib\sitepackages\tensorflow\python\platform\app.py", line 126, in run _sys.exit(main(argv)) File ".\lstm.py", line 67, in main Hist = Model.fit(x = X_train, y = Y_train, epochs = 300,batch_size=10, callbacks=[tensorboard], validation_data=(X_eval,Y_eval)); File "C:\Program Files\Python36\lib\sitepackages\keras\engine\training.py", line 1630, in fit batch_size=batch_size) File "C:\Program Files\Python36\lib\sitepackages\keras\engine\training.py", line 1480, in _standardize_user_data exception_prefix='target') File "C:\Program Files\Python36\lib\sitepackages\keras\engine\training.py", line 123, in _standardize_input_data str(data_shape)) ValueError: Error when checking target: expected dense_3 to have shape (10, 1) but got array with shape (10, 14)

Tensorflow chatbot rnn model test issue
I am getting the following error while testing seq2seq rnn model for chatbot
Key embedding_attention_seq2seq/embedding_attention_decoder/attention_decoder/Attention_0/bias not found in checkpoint [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_HALF, DT_INT32, DT_HALF, DT_HALF, DT_HALF, ..., DT_HALF, DT_HALF, DT_HALF, DT_HALF, DT_HALF], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] [[Node: save/RestoreV2/_1 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_6_save/RestoreV2", tensor_type=DT_HALF, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Keras Seq2Seq NMT with attention gives "AttributeError: 'NoneType' object has no attribute '_inbound_nodes'" for Seq2Seq NMT network with attention
I am building an Seq2Seq NMT model in Keras and I see the following error when trying to build the model:
AttributeError: 'NoneType' object has no attribute '_inbound_nodes'
I can't figure out which
NoneType
object it is referring to. Here is the code snippet. What am I doing wrong, and how can I fix this?#Encoder Specification encoder_inputs = Input(shape=(None,)) encoder_embedding = Embedding(max_words, latent_dim, weights=[embedding_matrix], trainable=False, mask_zero=True) embedding_output = encoder_embedding(encoder_inputs) encoder1 =LSTM(latent_dim, return_state=True,return_sequences=True) #orig encoder2 =LSTM(latent_dim, return_state=True,return_sequences=True) encoder3 =LSTM(latent_dim, return_state=True,return_sequences=True) encoder_outputs1, state_h1, state_c1 = encoder1(embedding_output)#orig encoder_outputs2, state_h2, state_c2 = encoder2(encoder_outputs1) encoder_outputs3, state_h3, state_c3 = encoder3(encoder_outputs2) encoder_states1 = [state_h1, state_c1] encoder_states2 = [state_h2, state_c2] encoder_states3 = [state_h3, state_c3] encoder_states = [state_h1, state_c1,state_h2, state_c2,state_h3, state_c3] #Attention layers specification from keras.activations import tanh, softmax dec_units = 1024 W1 = Dense(dec_units) W2 = Dense(dec_units) V = Dense(1) from keras.backend import sum, expand_dims from keras.layers import Activation, Permute, Multiply hidden_with_time_axis = expand_dims(state_h3, 1) print("hidden_with_time_axis:", hidden_with_time_axis) w1_encoder_outputs = W1(encoder_outputs3) w2_state_h_outputs = W2(hidden_with_time_axis) score = Activation('tanh')(w1_encoder_outputs + w2_state_h_outputs ) print("score:", score.shape) v_score = V(score) print("v_score", v_score.shape) attention_weights = Activation('softmax')(v_score) print("attention_weights:", attention_weights) context_vector = Multiply()([attention_weights, encoder_outputs3]) print("context_vector:", context_vector.shape) #DECOCER specification from keras.layers import Concatenate, RepeatVector, Add decoder_inputs = Input(shape=(None,)) decoder_embedding = Embedding(max_words, latent_dim, weights=[embedding_matrix], trainable=False, mask_zero=True) dec_embed_out = decoder_embedding(decoder_inputs) decoder_lstm1 = LSTM(latent_dim, return_sequences=True, return_state=True) decoder_lstm2 = LSTM(latent_dim, return_sequences=True, return_state=True) decoder_lstm3 = LSTM(latent_dim, return_sequences=True, return_state=True) print("context_vector:", context_vector.shape) print("dec_embed_out:", dec_embed_out.shape) concat_layer = Concatenate(axis=1) context_embed = concat_layer([context_vector,dec_embed_out]) print("conetxt_embed:", context_embed.shape) decoder_outputs1, _, _ = decoder_lstm1(context_embed, initial_state=encoder_states[0:2]) decoder_outputs2, _, _ = decoder_lstm2(decoder_outputs1, initial_state=encoder_states[2:4]) decoder_outputs3, _, _ = decoder_lstm3(decoder_outputs2, initial_state=encoder_states[4:6]) model = Model([encoder_inputs, decoder_inputs], context_embed)
Error Message:
/usr/local/lib/python3.6/distpackages/keras/engine/network.py in build_map(tensor, finished_nodes, nodes_in_progress, layer, node_index, tensor_index) 1323 ValueError: if a cycle is detected. 1324 """ > 1325 node = layer._inbound_nodes[node_index] 1326 1327 # Prevent cycles. AttributeError: 'NoneType' object has no attribute '_inbound_nodes'

How to make output sequence length equal to the input sequence length in tensorflow seq2seq model?
I am trying to formulate a resource mapping problem using seq2seq model where the input sequence length varies and the output sequence length must be equal to the input sequence length. From the manual of seq2deq in tensorflow, it seems that the output sequence length is determined by the end token or the maximal time step. So how can I make output length exactly equal to input length when building the seq2seq model?
tensorflow version: 1.11.0