Concatenating vectors for CNN in Keras/tensorflow
I am trying to concatenate the flattened output of a CNN, and a vector of scalar values. I am trying to influence the network by both the image and the vector. So the output of the flattened CNN size is (1,1024)
, and the vector I want to concatenate is (1,5)
.
Of course keras wants them to be similar sizes. So what is the best practice in this situation? Expand the vector to 1024
with zeroes? I am trying to make the vector have an impact, what other options can I do? I am trying to do something similar to here (https://arxiv.org/abs/1603.02199).
1 answer

Why not concatenating them over the last dimension, to obtain a tensor of shape
(1, 1029)
?from keras.models import Model from keras.layers import Input, Concatenate img = Input(shape=(1,1024)) vec = Input(shape=(1,5)) res = Concatenate(axis=1)([img, vec]) model = Model(inputs=[img, vec], outputs=res) model.summary() # _______________________________________________________________________________ # Layer (type) Output Shape Param # Connected to # =============================================================================== # input_1 (InputLayer) (None, 1, 1024) 0 # _______________________________________________________________________________ # input_2 (InputLayer) (None, 1, 5) 0 # _______________________________________________________________________________ # concatenate_1 (Concatenate) (None, 1, 1029) 0 input_1[0][0] # input_2[0][0] # =============================================================================== # Total params: 0 # Trainable params: 0 # Nontrainable params: 0 # _______________________________________________________________________________