Categories
Misc

How to use a consecutive sequence of one channel images to predict next frame label with Conv1D and LSTM?

Hi,

I am quite new to temporal forecast with images and LSTM. I
really appreciate your help.

Input is a sequence of images where each image size is 28*28,
and the number of this sequence of images is set as the batch size
as None the first argument of input_shape.

suppose 4 consecutive seconds of images were fed into the NN,
and the expected output of the NN would be No. 5 second label.

But I have hard time making Conv1D and LSTM working together and
ending up with one numerical label.

model = Sequential() model.add(Conv1D(40,2, strides=2,padding='same', activation='relu', input_shape=(None,28,28))) model.add(Reshape((None,576))) # or model.add(Flatten()) model.add(LSTM(10, activation='relu',stateful=True,return_sequences=True)) 
  1. Is the Batch size set properly?
  2. how to make Conv1D and LSTM linked together? I mean the data
    dimensionality stuff. Is it necessary to get the numerical labels
    from Conv1D and then pass them to LSTM? or pass the original
    dimensional data directly from Conv1D to LSTM then from the LSTM
    result, to compute one numerical label as the final result of
    NN?
  3. also is TimeDistributed() layer needed?

Thank you so much!

submitted by /u/boydbuilding

[visit reddit]

[comments]

Leave a Reply

Your email address will not be published. Required fields are marked *