I want to get mutual information in iris dataset to select best feature but i confused about mutual information.
You do not really understand something unless you can explain it to your grandmother.
Albert Einstein
Topic mutual-information feature-selection
Category Data Science
Mutual information (MI) quantifies the amount of information needed to express one variable with the help of another variable. If two variables are independent of each other, their MI is zero, else it will be greater than zero. MRMR, a mutual information based feature selection uses MI, which considers a feature effective if it has maximum MI with its class label(maximum relevance) and minimum MI with rest of the features(minimum redundancy). This combination of maximum relevance and minimum redundancy will ensure better performance with smaller feature dimension.
In simple terms, mutual information is used to measure relevance and redundancy of features with primary objective of high accuracy, with least possible time complexity (computational loss). Generally either a filter or a wrapper approach is taken for feature extraction. A good understanding of Cross-Entropy and Information Theory shall be helpful to understand MI concept. This paper has a proposed feature subset selection algorithm, based on (conditional) mutual information (along with LNC estimator) computation that you may quickly run through.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.