Understanding math notation in infoGAN paper

I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I kinda don't understand some of the code in the cost function. So, I looked at the paper to dissect it for better understanding and came across some math notation that I don't understand (pic below). The usage of the notations I'm trying to figure out are how the ~ in the Expectation subscripts, ', and || symbols are used.

This is what I think the notation means.

  • ~ in the expectation subscript: the variable on the left of ~ can be any continuous value coming from whatever is to the right of the ~
  • ' next to the c in P(c'|x): I have no clue. I thought those were symbols for derivates but that makes no sense for this equation so it's def not that.
  • ||: I'm not sure. I only know of these symbols being used in Norms but that's obviously not the case here.

Screenshot of a formula from the paper

The actual code I was trying to figure out was this in PyTorch. It's the variational lower bound (mutual information term) in the cost (It's not the formulas in the screenshot above). However, I ran into the formulas in the screenshot first before getting to the formula calculations for the variational lower bound formula.

mutual_information_lower_bound = lambda c_true, mean, logvar: Normal(mean,logvar.exp()).log_prob(c_true).mean()

Thanks for the help!

Topic gan mutual-information descriptive-statistics statistics

Category Data Science

1
answered at

  • The notation $D_\mathrm{KL}(P \| Q)$ is pretty much standard for the Kullback-Leibler divergence. There is an interesting discussion on Mathematics SE on the reasons for the fairly unusual notation used for divergences.

  • In general, $x \sim G(z, c)$ means "$x$ is a random variable distributed according to $G(z, c)$". In the subscript the author means to take the expectation with respect to the random variable $x$, supposing that $x$ has that distribution.

  • If you have a variable, such as $a$, it is not unusual to call a related variable $a'$ (usually pronounced "a prime"). In this case $c'$ is just a random variable distributed according to $P(c\mid x)$, with the name $c'$ probably meant to remind you that the variable is in some way connected to $c$. In cases where derivatives are obviously not the right thing, this is probably what a prime symbol means.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.