In case you don’t know what a confusion matrix is, is a tool that we use in machine learning to visualize the performance of a classification algorithm.

This kind of algorithms have some data, that can be in some class or not, and try to predict whether new data is in that class or not. For example, let’s say we have a model that classifies incoming mail as spam or not, we can use a confusion matrix to visualize how our original data is classified.

The confusion matrix is a contingency table of the training data with two dimensions: actual and predicted, i.e.: a 2x2 table with every possibility of correct prediction, incorrect prediction, actually in that class, and actually not in that class.

To facilitate its visualization, this table is usually coloured as a heatmap, where every square has a colour representing the “strength” of the value.

Nevertheless, all of this can be difficult to understand at a glance for people who are not into Machine Learning, and that’s why Shen et. al explored those alternatives.

Using bar charts

In the previously mentioned research paper, Hong et al. explore different representations of the same data and how they are understood by non-experts. From the results of their experiment, we can conclude that using a flow chart is the best way to make users who are not familiar with these plots understand them. Nevertheless, bar charts are better for simulating in your head, i.e.: calculating where a certain case will fall in the classes.

Furthermore, bar charts are easier to code and take up less space, so we decided to use them for a technical audience who might not be familiar with confusion matrices but surely is familiar with bar charts.

This figure highlights how there is a lot more spam than ham (non-spam).

Finally, without further ado, the Python code used to create this plot is:

_palette = sns.colour_palette('Paired')

_total = _toplot['count'].sum()
_max = _toplot['count'].max()
_toplot['pct'] = _toplot['count'] / _total
print(_toplot)
print("Total:", _total, "max:", _max)

with sns.axes_style("white"):
    # Get the colour map
    # https://stackoverflow.com/a/38575399/4505998
    g = sns.catplot(
        data=_toplot.reset_index(), y='count', col='Predicted', row='Actual',
        kind='bar', height=2, margin_titles=True, palette=_palette,
    )

g.set(
    # Set the ylim to the 120% to leave some space for letters
    ylim=(0,_max*1.2),
    # Remove left ticks (redundant)
    yticks=[],
)

# Setting the titles
for ax,l in zip(g.axes[:,0],g.row_names):
    ax.set_ylabel(l)
g.set_titles(col_template="{col_name}", row_template="")

# Making it more beautiful
g.despine(left=True)

colours = [5, 4, 2, 3]
for i, ax in enumerate(g.axes.ravel()):
    # Setting the bar labels
    # Display the number and percentage of proposals in each class
    for c in ax.containers:
        labels = [f'{v.get_height():.0f} ({v.get_height()/_total*100:.1f}%)' for v in c]
        ax.bar_label(c, labels=labels)

        # Set the colour of each bar to a colourmap
        for b in c:
            # plt.setp(b, colour=_cmap(b.get_height() / _max))
            # plt.setp(b, colour=_cmap(b.get_height() / _total))
            plt.setp(b, colour=_palette[colours[i]])
            
plt.savefig("bars.png")

References and Further reading

• Complete code with imports on Kaggle
• Hong Shen, Haojian Jin, Ángel Alexander Cabrera, Adam Perer, Haiyi Zhu, and Jason I. Hong. 2020. Designing Alternative Representations of Confusion Matrices to Support Non-Expert Public Understanding of Algorithm Performance. Proc. ACM Hum.-Comput. Interact. 4, CSCW2, Article 153 (October 2020), 22 pages. https://doi.org/10.1145/3415224
• Confusion Matrix - Wikipedia

Finally, if you liked this post, you might like other posts about data visualization like Creating a HeatMap with marginal distributions using Seaborn

For this, I used the JointGrid tool, which allows me to make a multi-plot grid for conditional relationships, where we can show the marginal distributions on the top and right parts of the graph.

The jointplot function has already many things built in, for plotting something like a scatter plot (with x and y variables) while showing its distribution.

In my case, what I wanted to plot wasn’t a scatter plot, It did not have an x nor an y, it was a ol’ heatmap created from a pd.crosstab.

This is my data:

array([[4850, 5713, 2401, 1715, 1523, 1328,  848],
       [1335, 5474, 6127, 3121, 2153, 1792, 1239],
       [1618,  874, 4309, 5731, 2849, 2117, 1381],
       [1802, 1353,  950, 4494, 5385, 2781, 1994],
       [2199, 1401,  989,  636, 4047, 4811, 2754],
       [2267, 1306,  973,  649,  542, 3161, 3462],
       [2988, 1452, 1035,  606,  602,  506, 2917]])

At first, I was going to create a Heatmap and two barplots, but then I decided to try to combine them all into one, and here is the result:

The code to accomplish this was the following:

# _toplot = pd.crosstab(...)

# Create an empty joint grid
grid = sns.JointGrid()

# Fill the centre with our heatmap
sns.heatmap(_toplot, ax=grid.ax_joint, cbar=False, annot=True, fmt='g', cmap='mako')
# Draw total bars, both with width 1, but the Y one with horizontal orientation
sns.barplot(_toplot.sum(), ax=grid.ax_marg_x, width=1)
sns.barplot(_toplot.sum(axis=1), ax=grid.ax_marg_y, orient='h', width=1)
# Offset value (just half an unit)
_off = .5

# Fix x
_xmin, _xmax = grid.ax_joint.get_xlim()
grid.ax_joint.set_xlim(_xmin+_off, _xmax+_off)
for bar in grid.ax_marg_x.containers[0]:
    bar.set_x(bar.get_x() + _off)

# Fix y
_ymin, _ymax = grid.ax_joint.get_ylim()
grid.ax_joint.set_ylim(_ymin+_off, _ymax+_off)
for bar in grid.ax_marg_y.containers[0]:
    bar.set_y(bar.get_y() + _off)

# Need to use this to set the horizontal_alignment
grid.ax_joint.set_xticklabels(
    grid.ax_joint.get_xticklabels(), 
    rotation=30,    
    ha='right'
)
grid.ax_joint.yaxis.set_tick_params(rotation=0)

grid.ax_joint.set_xlabel('Día de votación')
grid.ax_joint.set_ylabel('Día de creación')

Further reading

After coming up with this solution, I found this StackOverflow post:

• How to create a heatmap with marginal histograms, similar to a jointplot?

But the answer relies too much on modifying the data, and won’t work if the indexes are not a range of integers.

There’s also a discussion on GitHub about this which I replied to.