Changes in the standard Heatmap plot - symmetric bar colors, show only diagonal values, and column names at x,y axis ticks

I have a heatmap image (correlation between all matrix columns) and I'm straggling to preform all the changes below within the same image: bar colors should be symmetric around zero (e.g., correlation of 1 and -1 should be with the same color) change the correlation matrix to a diagonal matrix, since correlation values are symmetric - and show only upper matrix triangle (mask out the lower triangle ) show the correlation values in every cell of the diagonal matrix x,y …
Category: Data Science

Why grad cam is not showing despite of no error?

I am applying grad-cam on 3d images and I see no errors, I only can see the Scan of my original images but not the grad cam. from skimage.transform import resize from matplotlib import pyplot as plt capi=resize(cam,(79,95,79)) #print(capi.shape) capi = np.maximum(capi,0) heatmap = (capi - capi.min()) / (capi.max() - capi.min()) f, axarr = plt.subplots(2,3,figsize=(15,10)); f.suptitle('Grad-CAM') slice_count=78 slice_count2=20 axial_ct_img=np.squeeze(resized_img[slice_count, :,:]) axial_grad_cmap_img=np.squeeze(heatmap[slice_count,:, :]) coronal_ct_img=np.squeeze(resized_img[:,slice_count2,:]) coronal_grad_cmap_img=np.squeeze(heatmap[:,slice_count2,:]) img_plot = axarr[0,0].imshow(axial_ct_img, cmap='gray'); axarr[0,0].axis('off') axarr[0,0].set_title('CT') img_plot = axarr[0,1].imshow(axial_grad_cmap_img, cmap='jet'); axarr[0,1].axis('off') axarr[0,1].set_title('Grad-CAM') axial_overlay=cv2.addWeighted(axial_ct_img,0.5,axial_grad_cmap_img, 0.5,0, dtype = …
Category: Data Science

Dropping highly correlated features

I am making my classification project and I have this situation after using seaborn heatmap. Column 0 is my target, where I have data with 3 classes. To my knowledge I should remove column highly correlated with target value. My questions are: Should I remove also features highly correlated with features different than target? For example I can see very high correlation between column 27 and 46. Should I remove one of them? This is correlation heat map from my …
Category: Data Science

How to plot a heatmap-like plot for categorical features?

I would greatly appreciate let me know how to plot a heatmap-like plot for categorical features? In fact, based on this post, the association between categorical variables should be computed using Crammer's V. Therefore, I found the following code to plot it, but I don't know why he plotted it for "contribution", which is a numeric variable? def cramers_corrected_stat(confusion_matrix): """ calculate Cramers V statistic for categorical-categorical association. uses correction from Bergsma and Wicher, Journal of the Korean Statistical Society 42 …
Category: Data Science

GradCAM heatmap all negative

I've fully trained the VGG16 model on my dataset, resulting in a 97% validation accuracy. I'm using the code from this github: https://github.com/PowerOfCreation/keras-grad-cam but for 2 of my 16 classes, I'm getting all negative gradients, resulting in a heatmap with all null values. The other classes generate normal consistent maps which clear similar patterns. The problem is obviously occurring here because the 0 is always the max value. cam = np.maximum(cam, 0) heatmap = cam / np.max(cam) The only solution …
Category: Data Science

Sns Heatmap save as png jpeg image with full labels visible via python

I am trying to save my heat map in png format using the below code, but the labels are getting trimmed when I am saving the heatmap. Please help to keep the labels inside the figure. array = confusion_matrix(gts, preds, labels=class_name) #print(gts) #print(preds) df_cm = pd.DataFrame(array, index = class_name, columns = class_name) print("Confusion Matrix") print(df_cm) plt.figure(figsize = (10,7)) sns_plot = sns.heatmap(df_cm, annot=True, cmap =sns.cm.rocket_r,linecolor='white', linewidths=1) results_path = 'results.png' #print(results_path) plt.savefig(results_path, dpi=400)
Category: Data Science

Should I remove features such as gender and birth month before drawing the heatmap because they are categorical?

I am working on a dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is categorical (1, 2, 3) and I am performing EDA on this dataset. The categorical features with numerical values (ex: gender has values 0 and 1) are also considered when taking the heatmap with seaborn. As per my knowledge, the heatmap is drawn to check the correlation between continuous numerical features right (correct me if I am wrong). Should …
Category: Data Science

seaborn heatmap - x axis - repeated values

I'm in trouble creating a heatmap using a CSV file. csv data is in a format like below here is a code years = np.array(datadf.PublicationYear) sns.set(font_scale=2) wordlist = ['greenhouse_gas', 'pollution', 'resilience', 'urban','city', 'environmental_impacts', 'climate_change', 'adaptation','mitigation','carbon', 'ghg_emissions','sustainable','sustainability','lca'] word_tuples = [('urban','city','urban'), ('greenhouse_gas','ghg_emissions','greenhouse_gas'),('sustainable','sustainability','sustainability')] use_wordlist = True word_number = 30 freqdata = [] agg_keys = [] for i in np.arange(len(abstrct)): ngram_model = Word2Vec(ngram[[abstrct[i]]], size=100, min_count=1) ngram_model_counter = Counter() for key in ngram_model.wv.vocab.keys(): if key not in stoplist: if use_wordlist: if key in wordlist: if …
Category: Data Science

How to compare 10000 data frames in Python?

I have 10000 data frames like this: Each data frame corresponds to a different card game (and so different numbers in the table) and I want to compare these data frames all together. For example, I want to compare the heat maps of these data frames. Is there anyway to do this in Python? Is there any toolset so that I can compare all of them in one diagram or something like that? Because I want to see a trend …
Category: Data Science

Why do I get two different values in heatmap and feature_importances?

I'm running a feature selection using sns.heatmap and one using sklearn feature_importances. When using the same data I get two difference values. Here is the heatmap and heatmap code from matplotlib import pyplot as plt import pandas as pd import numpy as np import seaborn as sns training_data = pd.read_csv( "/Users/aus10/NFL/Data/Betting_Data/CBB/Training_Data_Betting_CBB.csv") df_model = training_data.copy() df_model = df_model.dropna() df_model = df_model.drop(['Money_Line', 'Money_Line_Percentage', 'Money_Line_Money', 'Money_Line_Move', 'Money_Line_Direction', "Spread", 'Spread_Percentage', 'Spread_Money', 'Spread_Move', 'Spread_Direction', "Win", "Money_Line_Percentage", 'Cover'], axis=1) X = df_model.loc[:, ['Total', 'Total_Move', 'Over_Percentage', 'Over_Money', …
Category: Data Science

Heatmap of large 2D array using datashader and plotly

I’m trying to show a heatmap of a large 2D array (160x250000 entries). This should go into a dash app so I'm using plotly to deal with graphics and my idea was to use datashader for performance but I’m having troubles getting it right. However, independently of dash I’m already having problem with plotly + datashader (see code below). There is probably something very basic I’m not understanding in this process. It would be great if someone could tell me …
Category: Data Science

Understanding CNN by visualizing class activations using GRAD_CAM

I followed the blog Where CNN is looking? to understand and visualize the class activations in order to predict something. The given example works very well. I have developed a custom model using autoencoders for image similarity. The model accepts 2 images and predicts the score for similarity. The model has the following layers: Layer (type) Output Shape Param # Connected to ================================================================================================== input_1 (InputLayer) (None, 256, 256, 3) 0 __________________________________________________________________________________________________ input_2 (InputLayer) (None, 256, 256, 3) 0 __________________________________________________________________________________________________ encoder …
Category: Data Science

Is converting a categorical value into numerical needed to find a correlation?

I have a small dataset of 1300 observations x 20 features. They are all numerical but one, which is categorical; this was calculated independently and relates to each observation in any case. I'm now attempting to find the correlation of each features in my dataset, but a simple dataframe.corr() would omit the categorical from the calculation. I have two choices as far as I can see: Do not consider the categorical value, but this means not being able to suggest …
Category: Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.