<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Data Visualization on David Davó&#39;s dev log</title>
    <link>https://blog.ddavo.me/tags/data-visualization/</link>
    <description>Recent content in Data Visualization on David Davó&#39;s dev log</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en</language>
    <copyright>© David Davó 2015 - 2024</copyright>
    <lastBuildDate>Fri, 14 Jun 2024 19:14:22 +0000</lastBuildDate><atom:link href="https://blog.ddavo.me/tags/data-visualization/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>On Spanish National Postal Service Crypto Stamps</title>
      <link>https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/</link>
      <pubDate>Fri, 14 Jun 2024 19:14:22 +0000</pubDate>
      
      <guid>https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/</guid>
      <description>In this post I briefly analyze how the crypto stamps recently released by Correos are implemented and used.</description>
      <content:encoded><![CDATA[<p>Very recently, Correos (Spain&rsquo;s national postal service) decided to launch a collection of &ldquo;crypto stamps&rdquo; based on Francisco Ibañez&rsquo;s famous character Mortadelo. Along with a special physical stamp that shows Mortadelo walking with his little case and all the usual gadgets, usable at a face value of 10€, you also get a code that can be redeemed for an NFT. All physical stamps have the same artwork, but the NFT shows one of 5 possible <em>criptosellos</em> (<em>crypto stamps</em>), each with a different print run and based on one of his classic disguises.</p>
<figure>
        <img loading="lazy" srcset="https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos_hufaaed248ba8d28c8bf7cb8138998d06c_207436_360x0_resize_q75_h2_box_2.webp 360w ,https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos_hufaaed248ba8d28c8bf7cb8138998d06c_207436_480x0_resize_q75_h2_box_2.webp 480w ,https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos_hufaaed248ba8d28c8bf7cb8138998d06c_207436_720x0_resize_q75_h2_box_2.webp 720w ,https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos_hufaaed248ba8d28c8bf7cb8138998d06c_207436_1080x0_resize_q75_h2_box_2.webp 1080w ,https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos_hufaaed248ba8d28c8bf7cb8138998d06c_207436_1500x0_resize_q75_h2_box_2.webp 1500w ,https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos.webp 2502w" 
            sizes="(min-width: 768px) 720px, 100vw" src="https://blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/todos-criptosellos.webp" alt="The 5 possible stamps are shown, based on 5 Mortadelo disguises" 
            width="2502" height="743"><figcaption>
            <p>The 5 possible Crypto Stamps and their print runs. / <i>
                    <a href="https://fesofi.es/noticias/primer-criptosello-espanol/">Spanish Federation of Philatelic Societies</a></p></i>
        </figcaption>
</figure>

<p>Broadly speaking, an NFT (Non-Fungible Token) is a <em>representation</em> of an asset (digital or physical) implemented on a blockchain. Even though it uses the same technology as cryptocurrencies, unlike cryptocurrencies NFTs are not liquid and cannot be divided. For example, 5 euros are still 5 euros whether they are banknotes, coins, a check, or your account balance. NFTs, however, represent indivisible assets (non-fungible). They can represent unique one-of-a-kind items, like the deed to your house that <em>represents</em> your house, or repeatable items with limited supply, like collectible cards in a set, or <strong>stamps</strong> in this case. Remember that an NFT is just a <em>token</em>, a <em>symbol</em>, a <em>ticket</em>. It is not <strong>the house</strong>, it is <strong>the deed</strong>. The value proposition of NFTs is that strong blockchain cryptography proves you are the real owner. So, instead of <em>a notary</em> or a government registry, you get a decentralized network plus a bunch of cryptographic algorithms proving it.</p>
<p>Blockchain has a bad reputation because it <a href="https://es.cointelegraph.com/news/in-2023-bitcoin-mining-consumed-more-than-one-third-of-mexicos-electrical-energy">can consume more electricity than countries such as Argentina</a>, but that is basically a Bitcoin issue. Ethereum no longer has that problem since the Ethereum 2.0 transition a few years ago. Also, Correos&rsquo; NFT runs on an alternative network called Polygon, a chain similar to Ethereum but much faster and more efficient (including energy efficiency).</p>
<p>In that sense, Correos did not do too badly. But the two main reasons to use blockchain are <strong>transparency</strong> and <strong>decentralization</strong>. In the rest of this post I will analyze these crypto stamps from those perspectives, and finally whether people are actually using them, and how much.</p>
<h2 id="transparency">Transparency</h2>
<p>This already starts poorly: the bare minimum transparency you should expect from something blockchain-based is publishing the smart contract address, and they did not do that.</p>
<p>Not publishing the contract address is like saying you have a transparency portal but not saying where it is. It hides the essential information users need to interact with the contract.</p>
<p>As someone who researches this stuff, I managed to find the contract on my own. Its address was <a href="https://polygon.blockscout.com/token/0x8fA19F4316dF640e14b30B2fE749cbdd79Da9335"><code>0x8fA19F4316dF640e14b30B2fE749cbdd79Da9335</code></a>.</p>
<blockquote>
<p>Update June 14, 2024: Some weird stuff happened and they <em>migrated</em> the contract <a href="https://polygon.blockscout.com/token/0x4D5D215c47a43B38a58Abf735b628cF9b61273a0">to another address</a>, in a rather clumsy way.</p>
</blockquote>
<p>They also do not publish the smart contract source code, so users cannot audit how it works or whether it is implemented correctly. There could be a bug that allows someone to grab all crypto stamps, and we might never know.</p>
<p>Even if you do not know programming, a common practice in this ecosystem is paying someone to audit contract code, yours or someone else&rsquo;s.</p>
<p>At least blockchain transfers are transparent, so we can still see who owns these exclusive Mortadelo NFTs.</p>
<h2 id="decentralization">Decentralization</h2>
<p>On the <strong>decentralization</strong> side, the solution is also quite disappointing. Normally NFTs use decentralized standards that let you use them across the wider web3 ecosystem: set one as your social media profile picture, show it in a virtual gallery, use it in games, and so on.
With these crypto stamps, you must use the Correos web app to view the associated image. When the consultancy firm maintaining that website drops support in a couple of years, you may lose access to your crypto stamp.</p>
<p><strong>If crypto stamps are neither transparent nor decentralized, why do they exist?</strong></p>
<p>I guess to increase stamp value and sell more units, right? We do not know how many stamps have been sold, but we can track how many were &ldquo;redeemed&rdquo; (or &ldquo;claimed&rdquo;) on-chain, and what kind of users did it.</p>
<blockquote>
<p>This post updates automatically with new data. Last update: 
    
    
        
        
    <span id="criptosellos-last-update" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/general-data.json" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/general-data.json">
        <span class="loading-text-data"></span>
</span>.</p>
</blockquote>
<p><em>Drum roll</em>&hellip; a total of <span id="criptosellos-total-tokens" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/general-data.json" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/general-data.json">
        <span class="loading-text-data"></span>
</span> crypto stamps have been claimed,
an average of <span id="criptosellos-daily-avg" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/general-data.json" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/general-data.json">
        <span class="loading-text-data"></span>
</span> per day.</p>


    
    
        
        
    
<figure data-page-path="/posts/dataviz/criptosellos-correos-mortadelo">
    <div class="chart" id="criptosellos-diario-chart"data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv"data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv">
        <div class="loading-chart-data"></div>
    </div><figcaption>
            Daily number of registered Crypto Stamps<p>Some days have very few claims, but there are occasional streaks. / <i>
                    <a href="https://dune.com/queries/3917833/6586501">David Davo via Dune Analytics.</a></p></i>
        </figcaption>
</figure>
<p>However, that does not mean there are as many people as stamps. In reality only <span id="criptosellos-total-holders" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv">
        <span class="loading-text-data"></span>
</span> different wallets hold at least one crypto stamp. We cannot claim these are unique users, because just like with email there is nothing stopping one person from creating multiple accounts. In crypto this is normal: keep one wallet with a very strong key for long-term storage, and another easier-to-use one (for example on your phone) so if it gets hacked, losses are limited.</p>
<p>Each user that holds at least one crypto stamp is called a <em>holder</em>. Although each <em>holder</em> has an average of <span id="criptosellos-holder-avg" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv">
        <span class="loading-text-data"></span>
</span> stamps, this is the classic &ldquo;if I eat two chickens and you eat none, on average we ate one each&rdquo; problem.</p>
<p>The reality is that one user has way more NFTs than everyone else: <span id="criptosellos-holder-whale-address" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv">
        <span class="loading-text-data"></span>
</span>, who holds <strong><span id="criptosellos-holder-whale-count" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv">
        <span class="loading-text-data"></span>
</span> crypto stamps</strong>. Meanwhile, most users (<span id="criptosellos-holder-oneortwo-cnt" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/daily-mints.csv">
        <span class="loading-text-data"></span>
</span>) hold one or two.</p>


    
    
        
        
    
<figure data-page-path="/posts/dataviz/criptosellos-correos-mortadelo">
    <div class="chart" id="criptosellos-holders-chart"data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv"data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv">
        <div class="loading-chart-data"></div>
    </div><figcaption>
            Number of crypto stamps per user<p>Users with few stamps (four or fewer) are grouped together. Hover to see the full address or user count. / <i>
                    <a href="https://dune.com/queries/3917785/6586366">David Davo via Dune Analytics.</a></p></i>
        </figcaption>
</figure>
<p>Another interesting question is whether these users had stamps from other collections. In other words, were they already familiar with this technology, or did Mortadelo bring them in?</p>
<p>Curiously, the wallet with the most crypto stamps had <strong>no</strong> previous NFTs, though maybe that person has another wallet with some; we can never know. Maybe it is a stamp seller from Plaza Mayor, who knows.</p>
<p>Overall, only <span id="criptosellos-holders-had-nfts-avg" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv">
        <span class="loading-text-data"></span>
</span> of users had ever held at least one NFT in their wallet. We do not even know whether they bought those NFTs or received them as gifts (or spam), so we cannot claim those few users are true <em>crypto enthusiasts</em>. What we can say confidently is that <strong><span id="criptosellos-holders-had-nfts-avg-not" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv">
        <span class="loading-text-data"></span>
</span> or more had never used NFTs before</strong>. What this means is that this product did not manage to attract crypto enthusiasts to the stamp collecting hobby, but the other way around.</p>
<blockquote>
<p>Note that I am checking not just Polygon (where these crypto stamps live), but all chains indexed in <a href="https://dune.com">Dune Analytics</a>.</p>
</blockquote>
<h2 id="money-talk">Money talk</h2>
<p>We all know crypto can move serious money, with <a href="https://forbes.es/criptomonedas/217739/la-crypto-punk-mania-este-es-el-top-10-de-colecciones-de-nft-de-2022/">some collections reaching millions of dollars</a>, but how much money do these crypto stamps move?</p>
<p>So far there have barely been any transfers, and none of them involved money through auction houses, so from that angle we cannot say whether owning a crypto stamp has any value beyond the physical stamp.</p>
<p>Still, we can estimate the <em>profit</em> this strategy has generated for Correos. Each unit costs 15€ and has a face value of 10€, meaning you can use it for postage up to 10€ and Correos keeps a 5€ markup.</p>
<p>Since <span id="criptosellos-total-tokens2" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv">
        <span class="loading-text-data"></span>
</span> stamps have been sold so far, Correos has made at least <span id="criptosellos-total-money" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/holders.csv">
        <span class="loading-text-data"></span>
</span> € in markup revenue.</p>
<p>Depending on whether the stamp is eventually used and for which postal product, real profit could be higher.</p>
<p>There are likely many stamps that were bought but not yet claimed on-chain. Still, I think it is fair to assume that if a buyer did not register it, then the crypto part added no value for that person.</p>
<p>Speaking of money: while I am not an expert in Spain&rsquo;s public procurement platform, it seems Correos has spent at least 
    
    
        
        
    <span id="criptosellos-correos-contratos" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/contratos.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/contratos.csv">
        <span class="loading-text-data"></span>
</span> EUR across <span id="criptosellos-correos-contratos-cnt" data-url="https://r2.blog.ddavo.me/posts/dataviz/criptosellos-correos-mortadelo/data/contratos.csv" data-local-url="/posts/dataviz/criptosellos-correos-mortadelo/data/contratos.csv">
        <span class="loading-text-data"></span>
</span> contracts to launch this collection. That said, part of that budget went into business-model research and framework design, so it could be amortized over future collections.</p>
<blockquote>
<p>You can download the procurement data I collected <a href="./contratos.csv">here</a>. If you spot anything wrong, please let me know.</p>
</blockquote>
<h2 id="conclusions">Conclusions</h2>
<p>In my humble opinion, Correos arrived very late and somewhat awkwardly to the 2021 NFT boom (already three years ago). On top of that, the lack of decentralization and transparency, two core values in this ecosystem, may make users less likely to buy crypto stamps.</p>
<p>Maybe they will recover the investment in future collections, hopefully with better implementation choices, because right now this is still far from what could reasonably be called web3. On the plus side, the website seems fairly accessible and works well.</p>
<p>That said, maybe this crossover has made at least one crypto nerd start collecting physical stamps and become a lifelong customer.</p>
<blockquote>
<p>Data was obtained through Dune Analytics, a blockchain data analysis platform. <a href="https://dune.com/ddavo/criptosellos">You can find the queries and datasets used here</a>.</p>
</blockquote>
]]></content:encoded>
    </item>
    
    <item>
      <title>Less confusing confusion matrices with Seaborn</title>
      <link>https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/</link>
      <pubDate>Wed, 22 May 2024 18:27:50 +0000</pubDate>
      
      <guid>https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/</guid>
      <description>In this post I use a JointGrid with bar plots to make confusion matrices less confusing</description>
      <content:encoded><![CDATA[<p>The other day my supervisor gave me an interesting paper to read on data visualization: <a href="https://dl.acm.org/doi/pdf/10.1145/3415224">Designing Alternative Representations of Confusion
Matrices to Support Non-Expert Public Understanding of
Algorithm Performance</a> (full reference below, on <em>further reading</em>).</p>
<p>In case you don&rsquo;t know what a confusion matrix is, is a tool that we use in machine learning to visualize the performance of a classification algorithm.</p>
<p>This kind of algorithms have some data, that can be in some class or not, and try to predict whether new data is in that class or not. For example, let&rsquo;s say we have a model that classifies incoming mail as spam or not, we can use a confusion matrix to visualize how our original data is classified.</p>
<p>The confusion matrix is a contingency table of the training data with two dimensions: <em>actual</em> and <em>predicted</em>, i.e.: a 2x2 table with every possibility of correct prediction, incorrect prediction, actually in that class, and actually not in that class.</p>
<p>To facilitate its visualization, this table is usually coloured as a <em>heatmap</em>, where every square has a colour representing the &ldquo;strength&rdquo; of the value.</p>
<figure>
        <img loading="lazy" srcset="https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/heatmap_hub7946e53d5e4abb803b2de6ac2038caf_15785_360x0_resize_box_3.png 360w ,https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/heatmap_hub7946e53d5e4abb803b2de6ac2038caf_15785_480x0_resize_box_3.png 480w ,https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/heatmap.png 640w" 
            sizes="(min-width: 768px) 720px, 100vw" src="https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/heatmap.png" alt="A sample confusion matrix with 47 true positives, 22 true negatives" 
            width="640" height="480"><figcaption>
            <p>A sample confusion matrix with 47 True Positives and 22 True Negatives (69 correct predictions) and 9 incorrect predictions</p>
        </figcaption>
</figure>

<p><strong>Nevertheless, all of this can be difficult to understand at a glance for people who are not into Machine Learning</strong>, and that&rsquo;s why Shen et. al explored those alternatives.</p>
<h2 id="using-bar-charts">Using bar charts</h2>
<p>In the previously mentioned research paper, Hong et al. explore different representations of the same data and how they are understood by non-experts. From the results of their experiment, we can conclude that using a flow chart is the best way to make users who are not familiar with these plots understand them. Nevertheless, bar charts are better for <em>simulating</em> in your head, i.e.: calculating where a certain case will fall in the classes.</p>
<p>Furthermore, bar charts are easier to code and take up less space, so we decided to use them for a technical audience who might not be familiar with confusion matrices but surely is familiar with bar charts.</p>
<figure>
        <img loading="lazy" srcset="https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/bars_hu062e503f807a7637a445da63ab0451cd_16724_360x0_resize_box_3.png 360w ,https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/bars.png 400w" 
            sizes="(min-width: 768px) 720px, 100vw" src="https://blog.ddavo.me/posts/tutorials/confusing-confusion-matrices-seaborn/bars.png" alt="A sample confusion matrix with 47 true positives and 22 true falses" 
            width="400" height="400"><figcaption>
            <p>The same data, but with another way of displaying it</p>
        </figcaption>
</figure>

<p>This figure highlights how there is a lot more spam than ham (non-spam).</p>
<p>Finally, without further ado, the Python code used to create this plot is:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">_palette</span> <span class="o">=</span> <span class="n">sns</span><span class="o">.</span><span class="n">colour_palette</span><span class="p">(</span><span class="s1">&#39;Paired&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">_total</span> <span class="o">=</span> <span class="n">_toplot</span><span class="p">[</span><span class="s1">&#39;count&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">_max</span> <span class="o">=</span> <span class="n">_toplot</span><span class="p">[</span><span class="s1">&#39;count&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">max</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">_toplot</span><span class="p">[</span><span class="s1">&#39;pct&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">_toplot</span><span class="p">[</span><span class="s1">&#39;count&#39;</span><span class="p">]</span> <span class="o">/</span> <span class="n">_total</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">_toplot</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="s2">&#34;Total:&#34;</span><span class="p">,</span> <span class="n">_total</span><span class="p">,</span> <span class="s2">&#34;max:&#34;</span><span class="p">,</span> <span class="n">_max</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">with</span> <span class="n">sns</span><span class="o">.</span><span class="n">axes_style</span><span class="p">(</span><span class="s2">&#34;white&#34;</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># Get the colour map</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># https://stackoverflow.com/a/38575399/4505998</span>
</span></span><span class="line"><span class="cl">    <span class="n">g</span> <span class="o">=</span> <span class="n">sns</span><span class="o">.</span><span class="n">catplot</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">data</span><span class="o">=</span><span class="n">_toplot</span><span class="o">.</span><span class="n">reset_index</span><span class="p">(),</span> <span class="n">y</span><span class="o">=</span><span class="s1">&#39;count&#39;</span><span class="p">,</span> <span class="n">col</span><span class="o">=</span><span class="s1">&#39;Predicted&#39;</span><span class="p">,</span> <span class="n">row</span><span class="o">=</span><span class="s1">&#39;Actual&#39;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="n">kind</span><span class="o">=</span><span class="s1">&#39;bar&#39;</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">margin_titles</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">palette</span><span class="o">=</span><span class="n">_palette</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">g</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># Set the ylim to the 120% to leave some space for letters</span>
</span></span><span class="line"><span class="cl">    <span class="n">ylim</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="n">_max</span><span class="o">*</span><span class="mf">1.2</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># Remove left ticks (redundant)</span>
</span></span><span class="line"><span class="cl">    <span class="n">yticks</span><span class="o">=</span><span class="p">[],</span>
</span></span><span class="line"><span class="cl"><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Setting the titles</span>
</span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">ax</span><span class="p">,</span><span class="n">l</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">g</span><span class="o">.</span><span class="n">axes</span><span class="p">[:,</span><span class="mi">0</span><span class="p">],</span><span class="n">g</span><span class="o">.</span><span class="n">row_names</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="n">l</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">g</span><span class="o">.</span><span class="n">set_titles</span><span class="p">(</span><span class="n">col_template</span><span class="o">=</span><span class="s2">&#34;</span><span class="si">{col_name}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">row_template</span><span class="o">=</span><span class="s2">&#34;&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Making it more beautiful</span>
</span></span><span class="line"><span class="cl"><span class="n">g</span><span class="o">.</span><span class="n">despine</span><span class="p">(</span><span class="n">left</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">colours</span> <span class="o">=</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">ax</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">g</span><span class="o">.</span><span class="n">axes</span><span class="o">.</span><span class="n">ravel</span><span class="p">()):</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># Setting the bar labels</span>
</span></span><span class="line"><span class="cl">    <span class="c1"># Display the number and percentage of proposals in each class</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">ax</span><span class="o">.</span><span class="n">containers</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">        <span class="n">labels</span> <span class="o">=</span> <span class="p">[</span><span class="sa">f</span><span class="s1">&#39;</span><span class="si">{</span><span class="n">v</span><span class="o">.</span><span class="n">get_height</span><span class="p">()</span><span class="si">:</span><span class="s1">.0f</span><span class="si">}</span><span class="s1"> (</span><span class="si">{</span><span class="n">v</span><span class="o">.</span><span class="n">get_height</span><span class="p">()</span><span class="o">/</span><span class="n">_total</span><span class="o">*</span><span class="mi">100</span><span class="si">:</span><span class="s1">.1f</span><span class="si">}</span><span class="s1">%)&#39;</span> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">c</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">        <span class="n">ax</span><span class="o">.</span><span class="n">bar_label</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">labels</span><span class="o">=</span><span class="n">labels</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">        <span class="c1"># Set the colour of each bar to a colourmap</span>
</span></span><span class="line"><span class="cl">        <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">c</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">            <span class="c1"># plt.setp(b, colour=_cmap(b.get_height() / _max))</span>
</span></span><span class="line"><span class="cl">            <span class="c1"># plt.setp(b, colour=_cmap(b.get_height() / _total))</span>
</span></span><span class="line"><span class="cl">            <span class="n">plt</span><span class="o">.</span><span class="n">setp</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">colour</span><span class="o">=</span><span class="n">_palette</span><span class="p">[</span><span class="n">colours</span><span class="p">[</span><span class="n">i</span><span class="p">]])</span>
</span></span><span class="line"><span class="cl">            
</span></span><span class="line"><span class="cl"><span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="s2">&#34;bars.png&#34;</span><span class="p">)</span>
</span></span></code></pre></div><h2 id="references-and-further-reading">References and Further reading</h2>
<ul>
<li>Complete code with imports on <a href="https://www.kaggle.com/code/daviddavo/blog-less-confusing-confusion-matrices">Kaggle</a></li>
<li>Hong Shen, Haojian Jin, Ángel Alexander Cabrera, Adam Perer, Haiyi Zhu, and Jason I. Hong. 2020. <em>Designing Alternative Representations of Confusion Matrices to Support Non-Expert Public Understanding of Algorithm Performance.</em> Proc. ACM Hum.-Comput. Interact. 4, CSCW2, Article 153 (October 2020), 22 pages. <a href="https://doi.org/10.1145/3415224">https://doi.org/10.1145/3415224</a></li>
<li><a href="https://en.wikipedia.org/wiki/Confusion_matrix">Confusion Matrix - Wikipedia</a></li>
</ul>
<p>Finally, if you liked this post, you might like other posts about data visualization like <a href="https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/">Creating a HeatMap with marginal distributions using Seaborn</a></p>
]]></content:encoded>
    </item>
    
    <item>
      <title>Creating a HeatMap with marginal distributions using Seaborn</title>
      <link>https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/</link>
      <pubDate>Fri, 16 Feb 2024 16:34:58 +0000</pubDate>
      
      <guid>https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/</guid>
      <description>In this post I create a neat heatmap using Seaborn&amp;#39;s JointGrid without modifying the DataFrame, and using data from my master&amp;#39;s thesis&amp;#34;</description>
      <content:encoded><![CDATA[<p>In this short post I will just share the code on how I created a very neat Heatmap for my Master&rsquo;s Thesis using <a href="https://seaborn.pydata.org">Seaborn</a>. It took me a while but I&rsquo;m very proud of how it turned out.</p>
<p>For this, I used the <a href="https://seaborn.pydata.org/generated/seaborn.JointGrid.html#seaborn.JointGrid">JointGrid</a> tool, which allows me to make a multi-plot grid for conditional relationships, where we can show the marginal distributions on the top and right parts of the graph.</p>
<p>The <a href="https://seaborn.pydata.org/generated/seaborn.jointplot.html#seaborn.jointplot">jointplot</a> function has already many things built in, for plotting something like a scatter plot (with x and y variables) while showing its distribution.</p>
<figure>
            <img loading="lazy" src="https://seaborn.pydata.org/_images/jointplot_17_0.png" alt=""/> <figcaption>
            <p>Sample figure from Seaborn&rsquo;s official documentation</p>
        </figcaption>
</figure>

<p>In my case, what I wanted to plot wasn&rsquo;t a scatter plot, It did not have an x nor an y, it was a ol&rsquo; heatmap created from a <a href="https://pandas.pydata.org/docs/reference/api/pandas.crosstab.html">pd.crosstab</a>.</p>
<p>This is my data:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">array([[4850, 5713, 2401, 1715, 1523, 1328,  848],
</span></span><span class="line"><span class="cl">       [1335, 5474, 6127, 3121, 2153, 1792, 1239],
</span></span><span class="line"><span class="cl">       [1618,  874, 4309, 5731, 2849, 2117, 1381],
</span></span><span class="line"><span class="cl">       [1802, 1353,  950, 4494, 5385, 2781, 1994],
</span></span><span class="line"><span class="cl">       [2199, 1401,  989,  636, 4047, 4811, 2754],
</span></span><span class="line"><span class="cl">       [2267, 1306,  973,  649,  542, 3161, 3462],
</span></span><span class="line"><span class="cl">       [2988, 1452, 1035,  606,  602,  506, 2917]])
</span></span></code></pre></div><p>At first, I was going to create a Heatmap and two barplots, but then I decided to try to combine them all into one, and <strong>here is the result</strong>:</p>
<figure>
        <img loading="lazy" srcset="https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/download_hua9674c314d701374f65f1d9b70634b14_80081_360x0_resize_box_3.png 360w ,https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/download_hua9674c314d701374f65f1d9b70634b14_80081_480x0_resize_box_3.png 480w ,https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/download.png 655w" 
            sizes="(min-width: 768px) 720px, 100vw" src="https://blog.ddavo.me/posts/tutorials/seaborn-heatmap-marginal-distribution/download.png" alt="My own Heatmap with marginal distributions" 
            width="655" height="641"><figcaption>
            <p>A heatmap on when a proposal is created and when is voted in a certain voting system (to be published)</p>
        </figcaption>
</figure>

<p>The code to accomplish this was the following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># _toplot = pd.crosstab(...)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Create an empty joint grid</span>
</span></span><span class="line"><span class="cl"><span class="n">grid</span> <span class="o">=</span> <span class="n">sns</span><span class="o">.</span><span class="n">JointGrid</span><span class="p">()</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Fill the centre with our heatmap</span>
</span></span><span class="line"><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">heatmap</span><span class="p">(</span><span class="n">_toplot</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="p">,</span> <span class="n">cbar</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">annot</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">fmt</span><span class="o">=</span><span class="s1">&#39;g&#39;</span><span class="p">,</span> <span class="n">cmap</span><span class="o">=</span><span class="s1">&#39;mako&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="c1"># Draw total bars, both with width 1, but the Y one with horizontal orientation</span>
</span></span><span class="line"><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">barplot</span><span class="p">(</span><span class="n">_toplot</span><span class="o">.</span><span class="n">sum</span><span class="p">(),</span> <span class="n">ax</span><span class="o">=</span><span class="n">grid</span><span class="o">.</span><span class="n">ax_marg_x</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">sns</span><span class="o">.</span><span class="n">barplot</span><span class="p">(</span><span class="n">_toplot</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">),</span> <span class="n">ax</span><span class="o">=</span><span class="n">grid</span><span class="o">.</span><span class="n">ax_marg_y</span><span class="p">,</span> <span class="n">orient</span><span class="o">=</span><span class="s1">&#39;h&#39;</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="c1"># Offset value (just half an unit)</span>
</span></span><span class="line"><span class="cl"><span class="n">_off</span> <span class="o">=</span> <span class="mf">.5</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Fix x</span>
</span></span><span class="line"><span class="cl"><span class="n">_xmin</span><span class="p">,</span> <span class="n">_xmax</span> <span class="o">=</span> <span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">get_xlim</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">set_xlim</span><span class="p">(</span><span class="n">_xmin</span><span class="o">+</span><span class="n">_off</span><span class="p">,</span> <span class="n">_xmax</span><span class="o">+</span><span class="n">_off</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">bar</span> <span class="ow">in</span> <span class="n">grid</span><span class="o">.</span><span class="n">ax_marg_x</span><span class="o">.</span><span class="n">containers</span><span class="p">[</span><span class="mi">0</span><span class="p">]:</span>
</span></span><span class="line"><span class="cl">    <span class="n">bar</span><span class="o">.</span><span class="n">set_x</span><span class="p">(</span><span class="n">bar</span><span class="o">.</span><span class="n">get_x</span><span class="p">()</span> <span class="o">+</span> <span class="n">_off</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Fix y</span>
</span></span><span class="line"><span class="cl"><span class="n">_ymin</span><span class="p">,</span> <span class="n">_ymax</span> <span class="o">=</span> <span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">set_ylim</span><span class="p">(</span><span class="n">_ymin</span><span class="o">+</span><span class="n">_off</span><span class="p">,</span> <span class="n">_ymax</span><span class="o">+</span><span class="n">_off</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">for</span> <span class="n">bar</span> <span class="ow">in</span> <span class="n">grid</span><span class="o">.</span><span class="n">ax_marg_y</span><span class="o">.</span><span class="n">containers</span><span class="p">[</span><span class="mi">0</span><span class="p">]:</span>
</span></span><span class="line"><span class="cl">    <span class="n">bar</span><span class="o">.</span><span class="n">set_y</span><span class="p">(</span><span class="n">bar</span><span class="o">.</span><span class="n">get_y</span><span class="p">()</span> <span class="o">+</span> <span class="n">_off</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Need to use this to set the horizontal_alignment</span>
</span></span><span class="line"><span class="cl"><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">set_xticklabels</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">    <span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">get_xticklabels</span><span class="p">(),</span> 
</span></span><span class="line"><span class="cl">    <span class="n">rotation</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span>    
</span></span><span class="line"><span class="cl">    <span class="n">ha</span><span class="o">=</span><span class="s1">&#39;right&#39;</span>
</span></span><span class="line"><span class="cl"><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">yaxis</span><span class="o">.</span><span class="n">set_tick_params</span><span class="p">(</span><span class="n">rotation</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s1">&#39;Día de votación&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">grid</span><span class="o">.</span><span class="n">ax_joint</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s1">&#39;Día de creación&#39;</span><span class="p">)</span>
</span></span></code></pre></div><h2 id="further-reading">Further reading</h2>
<p>After coming up with this solution, I found this StackOverflow post:</p>
<ul>
<li><a href="https://stackoverflow.com/a/65921757/4505998">How to create a heatmap with marginal histograms, similar to a jointplot?</a></li>
</ul>
<p>But the answer relies too much on modifying the data, and won&rsquo;t work if the indexes are not a range of integers.</p>
<p>There&rsquo;s also a <a href="https://github.com/mwaskom/seaborn/discussions/3198">discussion on GitHub</a> about this which I replied to.</p>
]]></content:encoded>
    </item>
    
  </channel>
</rss>
