What is the difference between a "cell" and a "layer" within neural networks?

So I understand what layers are. If you have 5 layers in your model, your data basically gets transformed 5 times via 5 activation functions. The number of neurons within a layer dictate how many outputs a layer creates.

So what are cells? I never understood where cells come into play. Are they a collection of layers?

Per Wiki: https://en.wikipedia.org/wiki/Long_short-term_memory

If the orange are layers, then I would imagine each has a bunch of neurons. So a cell is a collection of layers and yellow stuff? I'm having trouble understanding where this cell fits into an overall NN architecture. I am used to the pictures with input layer - hidden layer - output layer. So where would the cell occur?

Topic terminology deep-learning neural-network

Category Data Science

1
answered at

In short

A cell is in essence a forward neural network consisting of neurons and so $cell_{RNN}\neq neuron_{NN}$.

Rationale

In the context of recurrent neural networks, a layer consists of cells e.g. LSTM cells. Although they seem to position similarly in terms of architecture (figure), there is a fundamental difference between an RNN_cell and a NN_neuron.

  1. The number of RNN cells is generally equal to the sequence length (e.g. number of words in a sentence) or embedding output dimension (figure below). Conversely, the number of neurons in forward neural networks can generally be of any number.
  2. Each RNN_cell acts much like a NN itself. Specifically in LSTMs, each cell consists of hidden state $W_{hidden}$ and cell state $W_{cell}$ matrices. These are used to store weights and perform transformations in the various gates (forget, input, output) of the LSTM cell.

In essence, the transformations that take place in each cell are matrix multiplications of $W_{hidden}$ and $W_{cell}$ much like those in a forward NN. And so, the size of $W_{hidden}$ and $W_{cell}$ must be equal to the number of neurons of that NN which essentially determine the memory of the LSTM cells and model itself.

Figure 1

1
answered at

If you have 5 layers in your model, your data basically gets transformed 5 times via 5 activation functions.

That is not completely true. For dense neural networks (for example), your layers are made by neurons that each has its own activation function (so you'll have more than 5 activation functions). Each neuron transforms your data.

Now what is a cell ?

A cell is a mathematical function that receives one or more input, does the math, and gives one or more outputs.

First some examples:

  1. neuron: the result is the output of an activation function applied to the weighted sum of the previous layer
  2. convolution cell: passes the result of a convolution kernel multiplied to a relative range of the cells of the immediately previous layer to the next layer
  3. a lot of other cell examples in this post

Why are they called "cells" ? Because those mathematical function are used in artifical neural networks (ANN) context.

Why do they have a name ? Because they work well in (some) ANN, so people share their new mathematical function that helps an ANN to converge given a specific problem.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.