How to draw a simple LSTM network

Question

I'm new to deep learning, I am learning LSTM for my PhD work. This is a simple LSTM network for sequence classification. This code is from MATLAB tutorial: layers = [sequenceInputLayer(1) lstmLayer(5,'OutputMode','last') fullyConnectedLayer(3) softmaxLayer classificationLayer];

For simplicity, the dimension of the input sequence is 1, there are 3 classes.

I am trying to draw the diagram of this network. This is my attempt:

Is this right? Should the LSTM blue units be connected? The orange units are the softmax layer, is there any symbol the should be on each unit (like ∑)? The same question for each layer? Should be any extra layer to represent the "classificationLayer"? The fullyConnectedLayer is represented implicitly in the full connection in the last layer, Do I need to add any extra layer for this? Any other comments, please?

Answer 1

When thinking of rnn/lstm/gru layer keep few points in mind.

1. What is your inputs size? In this case we have 5 word sentence so 5 input circle going into lstm layer with that every word value will be multiple by same weight value.
2. How many lstm cell will be present inside lstm layer? To keep it simple as we have 5 words we are keeping 5 lstm cell for each word then carry forwarding it's memory into next lstm cell.
3. How many output categories we have? The last lstm cell will be connected to the all 3 dense layer neurons and at each of the neurons softmax operation will happend just like normal fully connected neural network dense layer works.