I've seen 2 definitions of the perplexity metric: $PP = 2^{H(p)}$ and $PP = 2^{H(p, q)}$ If I'm understanding correctly, the first one only tells us about how confident the model is about its predictions, while the second one reflects the accuracy/correctness of the model's predictions. Am I correct? Which one do people actually refer to when they claim their language model achieved X perplexity in their papers?

On my corpora, I am running LDA with different settings (I experiment with different number of topics, different different ngrams and TFIDF or regular BOW). Now, I want to rank these setups to select one best topic model to continue working with. In order to rank them, I have calculated both the coherence value as well as the perplexity for all the different settings, as is done here. In the link, the number of topics is selected using the coherence …