Siamese Network - Sigmoid function to compute similarity score

I would like to augment the answer of @Shubham Panchal, since I feel the real issue is still not made explicit.

1.) $\alpha$ could also contain negative entries so that the sigmoid function maps to $(0,1)$.

2.) @Stefan J, I think you are absolutely correct: two identical embedding vectors would be mapped to $0.5$ while two vectors that differ would be mapped to (depending on $\alpha$) values towards $1$ or $0$, which is not what we want!

@Shubham Panchal mentions the Dense layer and provides a link to an implementation, which is correct.

Now to make it very clear and short, in the paper they forgot to mention that there is a bias!

So it should be $p = \sigma(b+ \sum_{j}\alpha_{j}|h_{1,L-1}^{(j)} - h_{2,L-1}^{(j)}|)$.

Let $\hat{h} := \begin{pmatrix}\hat{h}_{1} & \ldots & \hat{h}_{n}\end{pmatrix}^{T}$, where $\hat{h}_{j}:= |h_{1,L-1}^{(j)} - h_{2,L-1}^{(j)}|$.

Then we know that $\hat{h}_{i} \geq 0$ for all $i$. If you consider now the classification problem geometrically, then $\alpha$ defines a hyperplane that is used to separate vectors $\hat{h}$ close to the origin from vectors $\hat{h}$ further away from the origin. Note that for $\alpha = 1$, we have $\sum_{j}\alpha_{j}|h_{1,L-1}^{(j)} - h_{2,L-1}^{(j)}| = ||\hat{h}||_{1}$. Using $\alpha$ results thus in a weighting of the standard $1$-norm, $\sum_{j}\alpha_{j}|\hat{h}^{(j)}|$.

Already for $n=2$ you can see that you can have two classes where the hyperplane must not go through the origin. For example, let's say two images belong together, if $\hat{h}_{1} \leq c_{1}$ and $\hat{h}_{2} \leq c_{2}$. Now you can not separate those points from points with $\hat{h}_{1} > c_{1}$ or $\hat{h}_{2}> c_{2}$ using a hyperplane that contains the origin. Therefore, a bias is necessary.

Using the Dense layer in Tensorflow will use a bias by default, though, which is why the presented code is correct.

Your observation is correct @Stefan J, but did you observe that there is a Dense layer too? The absolute L1 distances are multiplied with the weights of the Dense layer. These outputs are then fed to the sigmoid function. If we are looking at a Keras implementation from One Shot Learning with Siamese Networks using Keras,

you see the last Dense layer.

Also, from the research paper,

The L1 distances are multiplied with $\alpha$ which is a training parameter. This sum is then passed on to the sigmoid $\sigma$ function. We may interpret these values of $\alpha$ as the weights of the last Dense layer. These weights get smaller after training.

Another obvious reason of a sigmoid function is to get similarity scores in ( 0, 1 ). The binary cross-entropy loss function is used with it.