<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Rails on MoskitoHero</title><link>/tags/rails/</link><description>Recent content in Rails on MoskitoHero</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Tue, 06 May 2025 00:00:00 +0000</lastBuildDate><atom:link href="/tags/rails/index.xml" rel="self" type="application/rss+xml"/><item><title>An introduction to Barley, the fast model serializer</title><link>/post/2025-05-06-an-introduction-to-barley-the-fast-model-serializer/</link><pubDate>Tue, 06 May 2025 00:00:00 +0000</pubDate><guid>/post/2025-05-06-an-introduction-to-barley-the-fast-model-serializer/</guid><description>How I built and open-sourced Barley, a fast and simple model serializer gem for Rails, born from real-world performance needs and benchmarked against the best in the Ruby ecosystem.</description><content:encoded><![CDATA[<div class="sect1">
<h2 id="the-story"><a class="anchor" href="#the-story"></a>The story</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The company I used to work for had an ageing API in v1 that was very poorly designed and slow in its implementation. It served 4 frontend platforms (a website, two mobile apps and an administration tool). Being in a startup environment, our team had to move fast, but that was at the expense of performance and maintainability. This was the situation when I arrived there.
Returning JSON objects was very cumbersome. It used a custom method, available in each model that built a nested Hash, based on the attributes you gave it. That turned out to be a mess with poor performance and concerning security implications. It was difficult to maintain, with odd variable naming and behavior, so we set out to get rid of it somehow and moved to a kind of serializer DSL that was gradually introduced to replace the old mess.
Later, we decided to tighten up our API with a new version to get rid of the legacy. A few requirements quickly emerged, among which:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>API V2 should be easy to maintain, with clear naming conventions and namespacing</p>
</li>
<li>
<p>Since it is an internal API, it should be tightly coupled to the intended use and platform</p>
</li>
<li>
<p>API V2 should use some sort of view DSL to generate the JSON payloads</p>
</li>
<li>
<p>API V2 should be as predictable as possible in terms of typing and formatting.</p>
</li>
<li>
<p>API V2 should be super fast by default. It should be the last thing we need to optimize.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>So I looked at the serializer DSL that we had designed for API v1.
The first thing I did was to benchmark it against other solutions. So I stripped it out of legacy / no longer needed features, moved it to a local gem, and started playing with a few benchmarks.
And it was <em>very fast</em> indeed ! This is the result of a <a href="https://github.com/MoskitoHero/active_model_serializers/tree/benchmarks">benchmark</a> I did back then:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="shell">Warming up <span class="nt">--------------------------------------</span>
<span class="c"># Omitted for conciseness</span>
Calculating <span class="nt">-------------------------------------</span>
ams                    67.770  <span class="o">(</span>± 1.9%<span class="o">)</span> i/s -    657.000  <span class="k">in  </span>10.042270s
jsonapi-rb            157.239  <span class="o">(</span>± 2.3%<span class="o">)</span> i/s -      1.521k <span class="k">in  </span>10.010257s
barley                 96.909  <span class="o">(</span>± 2.4%<span class="o">)</span> i/s -    945.000  <span class="k">in  </span>10.036316s
barley-cache            4.589k <span class="o">(</span>± 2.8%<span class="o">)</span> i/s -     44.620k <span class="k">in  </span>10.010004s
ams          eager     68.237  <span class="o">(</span>± 6.2%<span class="o">)</span> i/s -    592.000  <span class="k">in  </span>10.087602s
jsonapi-rb   eager    289.733  <span class="o">(</span>± 3.7%<span class="o">)</span> i/s -      2.805k <span class="k">in  </span>10.033912s
barley       eager    406.732  <span class="o">(</span>± 3.2%<span class="o">)</span> i/s -      3.922k <span class="k">in  </span>10.020076s
barley-cache eager    639.935  <span class="o">(</span>± 2.4%<span class="o">)</span> i/s -      6.300k <span class="k">in  </span>10.053710s
with 95.0% confidence

Comparison:
barley-cache      :     4589.3 i/s
barley-cache eager:      639.9 i/s - 7.17x  <span class="o">(</span>± 0.27<span class="o">)</span> slower
barley       eager:      406.7 i/s - 11.29x  <span class="o">(</span>± 0.49<span class="o">)</span> slower
jsonapi-rb   eager:      289.7 i/s - 15.83x  <span class="o">(</span>± 0.74<span class="o">)</span> slower
jsonapi-rb        :      157.2 i/s - 29.17x  <span class="o">(</span>± 1.08<span class="o">)</span> slower
barley            :       96.9 i/s - 47.37x  <span class="o">(</span>± 1.75<span class="o">)</span> slower
ams          eager:       68.2 i/s - 67.25x  <span class="o">(</span>± 4.69<span class="o">)</span> slower
ams               :       67.8 i/s - 67.72x  <span class="o">(</span>± 2.31<span class="o">)</span> slower
with 95.0% confidence

Calculating <span class="nt">-------------------------------------</span>
<span class="c"># Omitted for conciseness</span>

Comparison:
barley-cache      :      46090 allocated
barley-cache eager:     216790 allocated - 4.70x more
barley       eager:     354326 allocated - 7.69x more
jsonapi-rb   eager:     694430 allocated - 15.07x more
jsonapi-rb        :     926082 allocated - 20.09x more
ams          eager:    1068038 allocated - 23.17x more
barley            :    1102354 allocated - 23.92x more
ams               :    1299674 allocated - 28.20x more</code></pre>
</div>
</div>
<div class="paragraph">
<p>Which is quite nice 😎.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="sharing-is-caring"><a class="anchor" href="#sharing-is-caring"></a>Sharing is caring</h2>
<div class="sectionbody">
<div class="paragraph">
<p>So I made a gem out of our serializer and <a href="https://github.com/MoskitoHero/barley">published it on Github</a>. It is named Barley.
I have recently made the gem even faster (from v0.9.0 on) and benchmarked it against <a href="https://github.com/okuramasafumi/alba">Alba</a>, <a href="https://github.com/procore-oss/blueprinter">Blueprinter</a>, <a href="https://github.com/malomalo/turbostreamer">Turbostreamer</a>, <a href="https://github.com/distil/jserializer">Jserializer</a>, <a href="https://github.com/estepnv/fast_serializer">FastSerializer</a>, <a href="https://github.com/trailblazer/representable">Representable</a>, <a href="https://github.com/vasilakisfil/SimpleAMS">SimpleAMS</a>, <a href="https://github.com/rails-api/active_model_serializers">AMS</a>, <a href="https://github.com/rails/jbuilder">Rails JBuilder</a> and <a href="https://github.com/yosiat/panko_serializer">Panko</a>.
Here are the results of these new benchmarks:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="shell">ruby 3.4.3 <span class="o">(</span>2025-04-14 revision d0b7e5b6a0<span class="o">)</span> +YJIT +PRISM <span class="o">[</span>x86_64-linux]
Warming up <span class="nt">--------------------------------------</span>
<span class="c"># Omitted for conciseness</span>

Calculating <span class="nt">-------------------------------------</span>
                alba    401.119 <span class="o">(</span>± 1.7%<span class="o">)</span> i/s    <span class="o">(</span>2.49 ms/i<span class="o">)</span> -      2.016k <span class="k">in   </span>5.027350s
alba_with_transformation
                        256.123 <span class="o">(</span>± 1.2%<span class="o">)</span> i/s    <span class="o">(</span>3.90 ms/i<span class="o">)</span> -      1.300k <span class="k">in   </span>5.076497s
         alba_inline     22.749 <span class="o">(</span>± 8.8%<span class="o">)</span> i/s   <span class="o">(</span>43.96 ms/i<span class="o">)</span> -    114.000 <span class="k">in   </span>5.049973s
                 ams     16.923 <span class="o">(</span>± 5.9%<span class="o">)</span> i/s   <span class="o">(</span>59.09 ms/i<span class="o">)</span> -     85.000 <span class="k">in   </span>5.029061s
              barley    427.045 <span class="o">(</span>± 2.6%<span class="o">)</span> i/s    <span class="o">(</span>2.34 ms/i<span class="o">)</span> -      2.162k <span class="k">in   </span>5.066130s
        barley_cache    378.285 <span class="o">(</span>± 2.4%<span class="o">)</span> i/s    <span class="o">(</span>2.64 ms/i<span class="o">)</span> -      1.927k <span class="k">in   </span>5.097462s
         blueprinter    120.704 <span class="o">(</span>± 1.7%<span class="o">)</span> i/s    <span class="o">(</span>8.28 ms/i<span class="o">)</span> -    612.000 <span class="k">in   </span>5.071371s
     fast_serializer    142.347 <span class="o">(</span>± 1.4%<span class="o">)</span> i/s    <span class="o">(</span>7.03 ms/i<span class="o">)</span> -    720.000 <span class="k">in   </span>5.058924s
         jserializer    252.455 <span class="o">(</span>± 1.6%<span class="o">)</span> i/s    <span class="o">(</span>3.96 ms/i<span class="o">)</span> -      1.274k <span class="k">in   </span>5.047584s
               panko    510.262 <span class="o">(</span>± 4.7%<span class="o">)</span> i/s    <span class="o">(</span>1.96 ms/i<span class="o">)</span> -      2.585k <span class="k">in   </span>5.077381s
               rails    107.589 <span class="o">(</span>± 7.4%<span class="o">)</span> i/s    <span class="o">(</span>9.29 ms/i<span class="o">)</span> -    546.000 <span class="k">in   </span>5.104715s
       representable     50.264 <span class="o">(</span>± 6.0%<span class="o">)</span> i/s   <span class="o">(</span>19.89 ms/i<span class="o">)</span> -    252.000 <span class="k">in   </span>5.027987s
          simple_ams     36.584 <span class="o">(</span>± 5.5%<span class="o">)</span> i/s   <span class="o">(</span>27.33 ms/i<span class="o">)</span> -    184.000 <span class="k">in   </span>5.054347s
       turbostreamer    351.551 <span class="o">(</span>± 1.7%<span class="o">)</span> i/s    <span class="o">(</span>2.84 ms/i<span class="o">)</span> -      1.776k <span class="k">in   </span>5.053401s

Comparison:
               panko:      510.3 i/s
              barley:      427.0 i/s - 1.19x  slower
                alba:      401.1 i/s - 1.27x  slower
        barley_cache:      378.3 i/s - 1.35x  slower
       turbostreamer:      351.6 i/s - 1.45x  slower
alba_with_transformation:      256.1 i/s - 1.99x  slower
         jserializer:      252.5 i/s - 2.02x  slower
     fast_serializer:      142.3 i/s - 3.58x  slower
         blueprinter:      120.7 i/s - 4.23x  slower
               rails:      107.6 i/s - 4.74x  slower
       representable:       50.3 i/s - 10.15x  slower
          simple_ams:       36.6 i/s - 13.95x  slower
         alba_inline:       22.7 i/s - 22.43x  slower
                 ams:       16.9 i/s - 30.15x  slower

Calculating <span class="nt">-------------------------------------</span>
<span class="c"># Omitted for conciseness</span>

Comparison:
               panko:     259178 allocated
              barley:     633521 allocated - 2.44x more
       turbostreamer:     641760 allocated - 2.48x more
alba_with_transformation:     818181 allocated - 3.16x more
                alba:     818241 allocated - 3.16x more
         jserializer:     822281 allocated - 3.17x more
        barley_cache:     849521 allocated - 3.28x more
     fast_serializer:    1470121 allocated - 5.67x more
         blueprinter:    2297921 allocated - 8.87x more
         alba_inline:    2736041 allocated - 10.56x more
               rails:    2757857 allocated - 10.64x more
                 ams:    4713401 allocated - 18.19x more
       representable:    5151321 allocated - 19.88x more
          simple_ams:    9017033 allocated - 34.79x more</code></pre>
</div>
</div>
<div class="paragraph">
<p>With YJIT enabled, Barley really shines and is the first Hash serializer of the group.
There are <a href="https://github.com/MoskitoHero/barley/tree/main/benchmarks#readme">more benchmarks to be found in the Barley repository</a>.
Barley is very simple in its implementation and allocates very little. I believe that it is what makes it so fast. I’ll write another post on how I optimized it in version 0.9.0.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="the-features"><a class="anchor" href="#the-features"></a>The Features</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Barley provides:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>a clear DSL to define your serializer</p>
</li>
<li>
<p>inline / block sub-serializer definition</p>
</li>
<li>
<p>russian-doll caching</p>
</li>
<li>
<p>type-checking with <a href="https://dry-rb.org/gems/dry-types/main/">dry-types</a></p>
</li>
<li>
<p>context handling</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This is how Barley works:</p>
</div>
<div class="sect2">
<h3 id="in-the-model-you-want-to-serialize"><a class="anchor" href="#in-the-model-you-want-to-serialize"></a>In the model you want to serialize</h3>
<div class="paragraph">
<p>First, include the Barley::Serializable concern in your model.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># /app/models/user.rb</span>
<span class="k">class</span> <span class="nc">User</span> <span class="o">&lt;</span> <span class="no">ApplicationRecord</span>
  <span class="kp">include</span> <span class="no">Barley</span><span class="o">::</span><span class="no">Serializable</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Once you include this module, Barley will look for a UserSerializable class in your codebase, so you need to define one, typically in the app/serializable directory. It should inherit from Barley::Serializer</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># /app/serializers/user_serializer.rb</span>
<span class="k">class</span> <span class="nc">UserSerializer</span> <span class="o">&lt;</span> <span class="no">Barley</span><span class="o">::</span><span class="no">Serializer</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Once this is set up, the as_json method on your model will return what you have defined in your serializer class.
You can also chose to use a different serializer to render your data. You can use the serializer method in your model. This method also allows you to pass a cache argument.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># /app/models/user.rb</span>
<span class="k">class</span> <span class="nc">User</span> <span class="o">&lt;</span> <span class="no">ApplicationRecord</span>
  <span class="kp">include</span> <span class="no">Barley</span><span class="o">::</span><span class="no">Serializable</span>

  <span class="n">serializer</span> <span class="no">MyCustomSerializer</span><span class="p">,</span> <span class="ss">cache: </span><span class="kp">true</span>
  <span class="c1"># or</span>
  <span class="n">serializer</span> <span class="no">MyCustomSerializer</span><span class="p">,</span> <span class="ss">cache: </span><span class="p">{</span><span class="ss">expires_in: </span><span class="mi">1</span><span class="p">.</span><span class="nf">hour</span><span class="p">}</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Another way to achieve this is through the as_json method:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="no">User</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">as_json</span><span class="p">(</span><span class="ss">serializer: </span><span class="no">MyCustomSerializer</span><span class="p">,</span> <span class="ss">cache: </span><span class="p">{</span><span class="ss">expires_in: </span><span class="mi">1</span><span class="p">.</span><span class="nf">hour</span><span class="p">})</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Please note that Barley overrides the as_json method and does not provide it with standard only, include or except arguments. This is because the purpose of this serializer is precisely to do that job. The root argument works, though.</p>
</div>
</div>
<div class="sect2">
<h3 id="defining-the-serializer"><a class="anchor" href="#defining-the-serializer"></a>Defining the serializer</h3>
<div class="paragraph">
<p>Let’s go back to the serializer class. Barley provides a simple DSL that allows you to define your JSON in a readable and maintainable way. Here is an example definition that covers most use cases:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># /app/serializers/user_serializer.rb</span>
<span class="k">class</span> <span class="nc">UserSerializer</span> <span class="o">&lt;</span> <span class="no">Barley</span><span class="o">::</span><span class="no">Serializer</span>

  <span class="n">attributes</span> <span class="ss">id: </span><span class="no">Types</span><span class="o">::</span><span class="no">Strict</span><span class="o">::</span><span class="no">Integer</span><span class="p">,</span> <span class="ss">:name</span>

  <span class="n">attribute</span> <span class="ss">:email</span>
  <span class="n">attribute</span> <span class="ss">:value</span><span class="p">,</span> <span class="ss">type: </span><span class="no">Types</span><span class="o">::</span><span class="no">Coercible</span><span class="o">::</span><span class="no">Integer</span>

  <span class="n">many</span> <span class="ss">:posts</span>

  <span class="n">one</span> <span class="ss">:group</span><span class="p">,</span> <span class="ss">serializer: </span><span class="no">CustomGroupSerializer</span>

  <span class="n">many</span> <span class="ss">:related_users</span><span class="p">,</span> <span class="ss">key: :friends</span><span class="p">,</span> <span class="ss">cache: </span><span class="kp">true</span>

  <span class="n">one</span> <span class="ss">:profile</span><span class="p">,</span> <span class="ss">cache: </span><span class="p">{</span> <span class="ss">expires_in: </span><span class="mi">1</span><span class="p">.</span><span class="nf">day</span> <span class="p">}</span> <span class="k">do</span>
    <span class="n">attributes</span> <span class="ss">:avatar</span><span class="p">,</span> <span class="ss">:social_url</span>

    <span class="n">attribute</span> <span class="ss">:badges</span> <span class="k">do</span>
      <span class="n">object</span><span class="p">.</span><span class="nf">badges</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:display_name</span><span class="p">)</span>
    <span class="k">end</span>
  <span class="k">end</span>

<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>For type-checking to work, you need to define a Types module like this, somewhere in your codebase:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="k">module</span> <span class="nn">Types</span>
  <span class="kp">include</span> <span class="no">Dry</span><span class="o">.</span><span class="no">Types</span><span class="p">()</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="sect3">
<h4 id="1-attributes"><a class="anchor" href="#1-attributes"></a>1. Attributes</h4>
<div class="paragraph">
<p>They can be defined as an array with the plural attributes method. It can be an array of symbols, or an array of hashes in the key: TypeDefinition pattern, or a mix of both.
They can also be defined with a singular attribute method, that accepts a type keyword argument.
If type is included in an attribute’s definition, it will raise an error. You can use types to coerce types and define constraints. Please refer to the <a href="https://dry-rb.org/gems/dry-types/main/">dry-types</a> gem for more options.
If you give a block to the attribute method, you can also return anything you like. You have the object object at hand that refers to the serialized model.</p>
</div>
</div>
<div class="sect3">
<h4 id="2-associations"><a class="anchor" href="#2-associations"></a>2. Associations</h4>
<div class="paragraph">
<p>They are defined with the one or many methods.
It will use the default serializer if no argument is given. You can provide it with a serializer and a cache option.
These methods do not provide a type argument, as it makes no sense to do so.
You can also decide to give it a block, that allows you to define the association’s serializer inline.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="wrapping-it-up"><a class="anchor" href="#wrapping-it-up"></a>Wrapping it up</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The Barley gem’s goal is to provide a simple DSL to define your JSON payloads with a type-proof option, and render it in a very fast and efficient manner. It is probably not as feature-packed as other serializers available to the rails community, but it is a no-brainer, yet powerful option that you should consider if you are looking for performance and simplicity.
I’d be happy to see more projects starting to adopt it and would love the community to provide optimizations to make it even better.</p>
</div>
</div>
</div>
]]></content:encoded></item><item><title>Track and debug your ActiveRecord SQL queries with annotations</title><link>/post/2024-12-10-annotating-sql-queries-for-debugging-purposes/</link><pubDate>Tue, 10 Dec 2024 00:00:00 +0000</pubDate><guid>/post/2024-12-10-annotating-sql-queries-for-debugging-purposes/</guid><description>Embed contextual comments directly into your SQL queries using ActiveRecord’s annotate method, making log analysis and debugging far easier.</description><content:encoded><![CDATA[<div class="paragraph">
<p>I recently <a href="https://blog.saeloun.com/2020/01/29/rails-6-adds-activerecord-relation-annotate-for-adding-sql-comments-to-activerecord-relation-queries/">discovered</a> a handy feature <a href="https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#method-i-annotate">introduced in Rails 6</a>. This feature allows you to annotate your SQL queries with comments directly embedded in the query itself. I know, I’m a bit late to the party, but I suspect I’m not the only one, so here’s a quick rundown.</p>
</div>
<div class="paragraph">
<p>In our app, we have a deeply nested hash called <code>public_data</code> that’s often used to render a record’s JSON payload, along with its associations, in the controller.</p>
</div>
<div class="paragraph">
<p>This <code>public_data</code> hash is built from a mix of attributes and model methods. However, it also calls the <code>public_data</code> methods of associated models, which might, in turn, call back to the original model’s <code>public_data</code>. This circular complexity has been a challenge, and while we’ve started transitioning to the <a href="https://github.com/MoskitoHero/barley">Barley serializer gem</a> for better structure, we still rely on this method in parts of our codebase for backward compatibility.</p>
</div>
<div class="paragraph">
<p>So when <code>@user.public_data</code> is called in the controller, it often triggers a long chain of database queries. Tracking down where a specific relation is being called—​whether in <code>User</code>, <code>Profile</code>, or <code>Subscription</code>--can be daunting.</p>
</div>
<div class="paragraph">
<p>While Rails logs are helpful, they require sifting through the output to correlate the log entry with the SQL query. This becomes even trickier when debugging in remote environments and searching through extensive logs in tools like New Relic or CloudWatch.</p>
</div>
<div class="paragraph">
<p>Enter <code>ActiveRecord::Relation#annotate</code>…​</p>
</div>
<div class="paragraph">
<p>This simple method lets you add comments to your SQL queries, embedding contextual information directly within the query.</p>
</div>
<div class="paragraph">
<p>Here’s how it works:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="n">users</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">active</span><span class="p">.</span><span class="nf">where</span><span class="p">(</span><span class="ss">score: </span><span class="o">..</span><span class="mi">5</span><span class="p">).</span><span class="nf">annotate</span><span class="p">(</span><span class="s2">&#34;called from Admin::UsersController&#34;</span><span class="p">)</span>

<span class="c1">#&gt; SELECT * FROM `users` WHERE `last_visit_at` &gt;= &#39;2024-11-10 16:53:05&#39;</span>
<span class="c1">#  AND `score` &lt;= 5 /* called from Admin::UsersController */ ORDER BY `id` ASC</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>This provides a clear indication of where the query originated. You can also include additional context, such as the current locale, user, or order ID, depending on your needs.</p>
</div>
<div class="paragraph">
<p>For example, adding an annotation inside the <code>public_data</code> method on my <code>Subscription</code> model makes it much easier to trace where the query originated in the chain.</p>
</div>
<div class="paragraph">
<p>Another great feature is the ability to stack annotations. Let’s say we add an annotation in the <code>active</code> scope of the <code>User</code> model:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="k">class</span> <span class="nc">User</span> <span class="o">&lt;</span> <span class="no">ApplicationRecord</span>
  <span class="n">scope</span> <span class="ss">:active</span><span class="p">,</span> <span class="o">-&gt;</span> <span class="p">{</span> <span class="n">where</span><span class="p">(</span><span class="ss">last_visit_at: </span><span class="mi">1</span><span class="p">.</span><span class="nf">week</span><span class="p">.</span><span class="nf">ago</span><span class="o">..</span><span class="p">).</span><span class="nf">annotate</span><span class="p">(</span><span class="s2">&#34;user active in the past week&#34;</span><span class="p">)</span> <span class="p">}</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Calling the same query as before would generate:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="n">users</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">active</span><span class="p">.</span><span class="nf">where</span><span class="p">(</span><span class="ss">score: </span><span class="o">..</span><span class="mi">5</span><span class="p">).</span><span class="nf">annotate</span><span class="p">(</span><span class="s2">&#34;called from Admin::UsersController&#34;</span><span class="p">)</span>

<span class="c1">#&gt; SELECT * FROM `users` WHERE `last_visit_at` &gt;= &#39;2024-11-10 16:53:05&#39;</span>
<span class="c1">#  AND `score` &lt;= 5 /* user active in the past week */ /* called from Admin::UsersController */</span>
<span class="c1">#  ORDER BY `id` ASC</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Neat, right? This approach brings more clarity and structure to your logs, making them easier to query and group in log viewers. Additionally, these annotations appear in your database logs, offering a seamless way to correlate them with application logs.</p>
</div>
<div class="paragraph">
<p>Possible applications include:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Tracking queries in background jobs</p>
</li>
<li>
<p>Tagging controller context, headers, or anything else inside the queries</p>
</li>
<li>
<p>Debugging N+1 or slow queries</p>
</li>
<li>
<p>Expliciting the scope of queries</p>
</li>
<li>
<p>And so much more creative stuff!</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>That said, annotations come with a slight tradeoff in query readability. It’s essential to be deliberate about what and when to log, and to periodically review whether the annotations remain relevant and valuable.</p>
</div>
<div class="paragraph">
<p>Even if used exclusively for local debugging, SQL annotations are an excellent addition to your toolbox.</p>
</div>
]]></content:encoded></item><item><title>Lessons learned after migrating from Heroku to a kubernetes cluster</title><link>/post/2021-02-20-lessons-learned-after-migrating-from-heroku-to-a-kubernetes-cluster/</link><pubDate>Sat, 20 Feb 2021 00:00:00 +0000</pubDate><guid>/post/2021-02-20-lessons-learned-after-migrating-from-heroku-to-a-kubernetes-cluster/</guid><description>I recently migrated my company’s rails hosting from Heroku to a kubernetes cluster. It was a daunting task, but I learned a lot. Here are the lessons I learned, and how you can avoid the pitfalls I encountered.</description><content:encoded><![CDATA[<div class="paragraph">
<p>Part of the mission I was given when I arrived at my now workplace was the complete migration of our rails hosting from a Heroku service to a kubernetes solution. This was my first experience migrating a production stack, and my first experience with kubernetes as well. And just in case you’d think the task wasn’t daunting enough, the previous sole developper decided to leave for another opportunity a few weeks after my arrival. This departure was soon forgotten with the company hiring another developper who had already transitioned an app to kubernetes. So I took a course on udemy, read the kubernetes documentation thouroughly, took notes, and played around with a toy cluster. Then I set out on the transition journey.</p>
</div>
<div class="paragraph">
<p>Six months later, the Heroku-Kubernetes transition is now over. I have gained a lot of experience. And I feel it’s time to share it.</p>
</div>
<div class="sect1">
<h2 id="the-stack"><a class="anchor" href="#the-stack"></a>The stack</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Our company’s flagship product is a smart lamp that detects elderly people’s movements and activities and send an alarm whenever the fall. It is mainly used in french retirement houses, but we have received quite a lot of buzz in the silver economy field and we are about to scale worldwide, and sell our product, along with its web infrastructure, to franchised resellers. This is where Kubernetes comes as a handy solution, as it allows for quick replication and scaling of a single stack.</p>
</div>
<div class="paragraph">
<p>So the stack consists in a MQTT broker (hosted with CloudMQTT) that receives the messages sent by the smart lamps and their accessories. A sidekiq worker is then constantly polling the broker to get the messages and take action based on the content: send alerts, monitor activity, or just log temperature and so on. The rails web app is in charge of an admin interface and a few API endoints used by the mobile apps. Other connected services include Heroku Postgres for the data, Redis as the Sidekiq backend, Elasticsearch to index and search the logged MQTT payloads, logentries to track the rails log, Sentry for alerting and Sendgrid for emails.</p>
</div>
<div class="paragraph">
<p>All these services had been subscribed to as Saas from the Heroku app console.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="what-i-did-right"><a class="anchor" href="#what-i-did-right"></a>What I did right</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="i-asked-for-time"><a class="anchor" href="#i-asked-for-time"></a>I asked for time</h3>
<div class="paragraph">
<p>Given the context, that was the first thing I did. The company wanted a swift move, but I said I had to learn about the stack, get acquainted with the codebase and troubleshoot a few issues before I could get a good idea of what the implications of such a migration meant. They agreed, and gave me a few months to transition.</p>
</div>
<div class="paragraph">
<p>I can only imagine how painful it would have been to undertake such a task with a limited knowledge of the app.</p>
</div>
</div>
<div class="sect2">
<h3 id="i-asked-for-help"><a class="anchor" href="#i-asked-for-help"></a>I asked for help</h3>
<div class="paragraph">
<p>My previous devops-ish experience was limited to editing my Capistrano configuration and maintaining a few VPS. Pushing a new update at my new workplace merely consisted in typing <code>git push heroku master</code> and maybe run a migration on the server afterwards. Tuning the stack consisted in adding or removing a dyno, and changing the Saas billing plan to get more ressources.</p>
</div>
<div class="paragraph">
<p>When I started looking at the way Kubernetes is architectured, I quickly understood that I needed the help of a consultant to guide me and advise me on how to build the new stack and make wise decisions on the myriad of configuration options available. I am ever so thankful to that guy, who happened to be very friendly and patient. Kudos.</p>
</div>
</div>
<div class="sect2">
<h3 id="i-played-around-and-took-notes"><a class="anchor" href="#i-played-around-and-took-notes"></a>I played around and took notes</h3>
<div class="paragraph">
<p>This is important. I destroyed and rebuilt the cluster from scratch at least three times. Every time, I wrote down stuff in a wiki for reference. And every time, I improved the workflow and gained a lot of experience.</p>
</div>
<div class="paragraph">
<p>I deleted pods, services, recreated them, fiddled with PVCs, tried out many helm charts…​ This gave me hands-on ecxperience and allowed me to crash things a few times before knowing how to get things working.</p>
</div>
</div>
<div class="sect2">
<h3 id="i-improved-the-production-workflow"><a class="anchor" href="#i-improved-the-production-workflow"></a>I improved the production workflow</h3>
<div class="paragraph">
<p>Maybe I should say &#34;he&#34;, because that’s what the kubernetes consultant’s job.</p>
</div>
<div class="paragraph">
<p>We now have a nice, fully integrated CI flow that creates kubernetes environments for production, testing and staging. It still needs to be improved, but we are almost on par with <code>git push heroku master</code> now.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="what-i-did-wrong"><a class="anchor" href="#what-i-did-wrong"></a>What I did wrong</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Lots of things. Here are the details.</p>
</div>
<div class="sect2">
<h3 id="i-lost-time-adapting-the-stack"><a class="anchor" href="#i-lost-time-adapting-the-stack"></a>I lost time adapting the stack</h3>
<div class="paragraph">
<p>When I introduced Elasticsearch to the stack, our memory consumption went ballistic. So we decided to find an alternative. Elasticsearch is not used as an end-user-facing feature. It is used by our aftersales department and our developers. So we decided that searching the logs could take 3 seconds instead of 300ms and got rid of elasticsearch altogether.</p>
</div>
<div class="paragraph">
<p>We were wrong. Not only did I spend a lot of time researching solutions, tuning indices, partitioning tables, refactoring code, but in the end I didn’t come up with a viable alternative that could be used on a day-to-day basis by my fellow workers. So I decided to temporarily move back to Elasticsearch to transition, and do the reasearch later on. That should have been our decision from the start, because we lost plenty of man-hour money for the sake of saving a few yearly hundred bucks.</p>
</div>
</div>
<div class="sect2">
<h3 id="i-added-features"><a class="anchor" href="#i-added-features"></a>I added features</h3>
<div class="paragraph">
<p>When you are given a new toy, it is tempting to try it in every situation that pops about.</p>
</div>
<div class="paragraph">
<p>Don’t.</p>
</div>
<div class="paragraph">
<p>After a few weeks, the k8s branch had diverged from master in an tremendous measure. Again, it could have gone horribly wrong, and it did to some extent, as I had to do a few live patches on day 2 after a few customers explaind that some parts of the service was not behaving as expacted.</p>
</div>
</div>
<div class="sect2">
<h3 id="i-changed-versions"><a class="anchor" href="#i-changed-versions"></a>I changed versions</h3>
<div class="paragraph">
<p>Hey. Postgres 12 is available. Let’s use it.</p>
</div>
<div class="paragraph">
<p>Maybe that’s a good idea to upgrade your database. But just keep this for a later time, once the dust is settled. I did encounter a few bugs related to that upgrade. They were not gigormous bugs. But they could have been avoided, or postponed.</p>
</div>
</div>
<div class="sect2">
<h3 id="i-moved-everything-in-one-go"><a class="anchor" href="#i-moved-everything-in-one-go"></a>I moved everything in one go</h3>
<div class="paragraph">
<p>We decided that we wanted to make our cluster totally self-contained and stop relying on Saas, except for Sentry and Sendgrid. So we went for Stolon as a high-availability Postgresql database, VerneMQ for the MQTT broker, and bitnami helm charts for Elasticsearch and Redis.</p>
</div>
<div class="paragraph">
<p>This is fine. But it is too much to handle in a single move. I had quite a lot of experience with handling a Postgres database, Redis is OK, but I was a complete newbie to elasticsearch administration. On migration day, this was the servoce that caused the most issues, and I had to reindex the 80 million logs database twice to get it right without crashing the server.</p>
</div>
<div class="paragraph">
<p>Looking back, things could have gone extremely wrong.</p>
</div>
<div class="paragraph">
<p>I should have taken things gradually:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>First, the rails app, Redis and Postgres. That’s the core of our stack. And that’s enough work for a fist pass. Fix the bugs, tune the app, and let it settle for a week or two.</p>
</li>
<li>
<p>Second, set up Grafana. I use Loki for logs, so it is a nice replacement for Logentries.</p>
</li>
<li>
<p>Then, bring in the MQTT broker. It’s easy, because you can just bridge the messages from one to another. Then that’s just a case of pointing the devices to the new URL. That’s the IoT dev team’s problem, not mine.</p>
</li>
<li>
<p>Finally, try Elasticsearch with a subset of logs. See how it behaves. Then gradually move the rest to the new infrastrucure.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="tl-dr-how-you-should-do-it"><a class="anchor" href="#tl-dr-how-you-should-do-it"></a>TL; DR; How you should do it</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Don’t be too greedy, be humble and try moving things little by little. Try to keep the stack as similar as possible as the Heroku stack to start with: same software versions, same environment. Only then can you upgrade software if you wish to do so. Keep the connected Saas running, and gradually add them to your stack if that’s what you want to do. But be sure to have a proper testing and staging environment beforehand.</p>
</div>
</div>
</div>
]]></content:encoded></item><item><title>An alternative to the ActiveRecord after_initialize callbacks</title><link>/post/2020-10-08-an-alternative-to-the-activerecord-after-initialize-callbacks/</link><pubDate>Sat, 03 Oct 2020 00:23:33 +0000</pubDate><guid>/post/2020-10-08-an-alternative-to-the-activerecord-after-initialize-callbacks/</guid><description>Avoiding the callback hell in Rails by creating a simple method in the model to initialize associated records.</description><content:encoded><![CDATA[<div class="paragraph">
<p>I was playing with dynamic nested forms with Stimulus today and I stumbled upon a void left by Rails&#39; ActiveRecord callbacks.</p>
</div>
<div class="paragraph">
<p>I have this Caregiver model within a nested form and I need to create it along with a User entity in my form, to be able to populate the fields. The problem is that creating an <code>after_initialize</code> callback (even with <code>:new_record?</code> will break the general behaviour of my app, and I don’t want to run through all the specs and fix FactoryBot issues etc.</p>
</div>
<div class="paragraph">
<p>So I just decided to create a small method in my Caregiver model:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># caregiver.rb</span>

<span class="k">def</span> <span class="nf">init_with_user</span>
  <span class="nb">self</span><span class="p">.</span><span class="nf">build_user</span>
  <span class="nb">self</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Then I just need to call it in my view like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="erb"><span class="cp">&lt;%=</span> <span class="n">f</span><span class="p">.</span><span class="nf">fields_for</span> <span class="ss">:caregivers</span><span class="p">,</span> <span class="n">caregiver</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">init_with_user</span><span class="p">,</span> <span class="ss">child_index: </span><span class="s1">&#39;TEMPLATE_RECORD&#39;</span> <span class="k">do</span> <span class="o">|</span><span class="n">cf</span><span class="o">|</span> <span class="cp">%&gt;</span>
  <span class="cp">&lt;%=</span> <span class="n">render</span> <span class="s1">&#39;caregiver_fields&#39;</span><span class="p">,</span> <span class="ss">cf: </span><span class="n">cf</span> <span class="cp">%&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>This makes the code in my view simpler, more explicit, while avoiding the dreaded callback hell.</p>
</div>
]]></content:encoded></item><item><title>Chaining conditions in regular expressions</title><link>/post/2020-10-03-chaining-conditions-in-regular-expressions/</link><pubDate>Sat, 03 Oct 2020 00:23:33 +0000</pubDate><guid>/post/2020-10-03-chaining-conditions-in-regular-expressions/</guid><description>Unleash the full power of regular expressions by chaining conditions together.</description><content:encoded><![CDATA[<div class="imageblock">
<div class="content">
<img src="/assets/img/regexp.jpg" alt="Regexp hell"/>
</div>
</div>
<div class="paragraph">
<p>I recently had to make a semi-complex check on a string to validate it.</p>
</div>
<div class="paragraph">
<p>Basically, I have this model method which produces a serial number based on the ID of the object. Valid returned values for this method are <code>L00000002</code>, <code>L00000587</code> or <code>L00014522</code>. In short, they should be constituted of a capital <code>L</code> followed by 8 digits composed of the <code>id</code> of the object padded with zeros.</p>
</div>
<div class="paragraph">
<p>So I had to find a way to simply created a matcher for this to use in my model spec. I could have built two expectations, one for the length, and one for the composition. But I just decided to believe that regular expressions are powerful enough to do both at the same time. So I headed to <a href="https://rubular.com">rubular</a> and started fiddling and googling around.</p>
</div>
<div class="paragraph">
<p>The solution I came up with was the following (with an <code>id</code> of <code>12</code>, for instance):</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="regexp">/^L(?=[0-9]{8}$)0*12$/</code></pre>
</div>
</div>
<div class="paragraph">
<p>What this does, is - after the starting <code>^L</code> character - create a condition matching whatever follows the <code>?=</code> inside the parentheses. If this condition is true, then the rest of the expression is read. So I just check for any digit (<code>[0-9]</code> - it could have been <code>\d</code> but readability make the first option a better fit IMHO) exacty 8 times (<code>{8}</code>) and then the end of the string with <code>$</code>. If this is matched, then I check any number of zeros (<code>0*</code>) followed by the id (<code>12</code>) and the end of the string (<code>$</code>).</p>
</div>
<div class="paragraph">
<p>Now I could use this expression in my spec, using the <code>Regexp.escape</code> method on my model’s <code>id</code> within a string interpolation:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># spec/models/my_model_spec.rb</span>
<span class="n">describe</span> <span class="s2">&#34;serial_number&#34;</span> <span class="k">do</span>
  <span class="n">let</span><span class="p">(</span><span class="ss">:my_model</span><span class="p">)</span> <span class="p">{</span> <span class="n">create</span><span class="p">(</span><span class="ss">:my_model</span><span class="p">)</span> <span class="p">}</span> <span class="c1"># I use the FactoryBot gem</span>

  <span class="n">it</span> <span class="s2">&#34;returns a valid serial number&#34;</span> <span class="k">do</span>
    <span class="n">expect</span><span class="p">(</span><span class="n">my_model</span><span class="p">.</span><span class="nf">serial_number</span><span class="p">).</span><span class="nf">to</span> <span class="n">match</span><span class="p">(</span><span class="sr">/^L(?=[0-9]{8}$)0*</span><span class="si">#{</span><span class="no">Regexp</span><span class="p">.</span><span class="nf">escape</span><span class="p">(</span><span class="n">my_model</span><span class="p">.</span><span class="nf">id</span><span class="p">.</span><span class="nf">to_s</span><span class="p">)</span><span class="si">}</span><span class="sr">$/</span><span class="p">)</span>
  <span class="k">end</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="olist lowerroman">
<ol class="lowerroman" type="i">
<li>
<p>and create the corresponding method like so:</p>
</li>
</ol>
</div>
<div class="listingblock">
<div class="content">
<pre># app/models/my_model.rb
def serial_number
  &#34;L#{id.to_s.rjust(8, &#34;0&#34;)}&#34;
end</pre>
</div>
</div>
<div class="paragraph">
<p>Don’t you just love regular expressions?</p>
</div>
]]></content:encoded></item><item><title>Prevent Devise from sending confirmation emails</title><link>/post/2020-01-26-prevent-devise-from-sending-confirmation-emails/</link><pubDate>Sun, 26 Jan 2020 00:00:00 +0000</pubDate><guid>/post/2020-01-26-prevent-devise-from-sending-confirmation-emails/</guid><description>When you need to import users into a new Rails app using Devise, you may want to skip sending confirmation emails. Here’s how to do it by overriding an instance method in the User model.</description><content:encoded><![CDATA[<div class="imageblock">
<div class="content">
<img src="/assets/img/message-in-a-bottle.jpg" alt="Image par Antonios Ntoumas de Pixabay"/>
</div>
</div>
<div class="paragraph">
<p>I am about to release a huge rails project I have been working on for quite a
while. This project is meant to replace an old app that manages a database of
customers with emails. The new rails project uses Devise for authentication.</p>
</div>
<div class="paragraph">
<p>Before we push the final update to the production server, we need to find a way to import the users without sending confirmation emails, but still keep them unconfirmed until they actually login to the new system and provide a password. New registrations should also still be confirmable.</p>
</div>
<div class="paragraph">
<p>So I had to find a way to override an instance method. This is easy to do with a
simple module:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="c1"># Module to be included in a User instance whenever we don&#39;t want to send an email</span>
<span class="c1"># Useful when importing users or for testing</span>
<span class="k">module</span> <span class="nn">SkipConfirmationEmail</span>
  <span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">included</span><span class="p">(</span><span class="n">base</span><span class="p">)</span>
    <span class="n">base</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:include</span><span class="p">,</span> <span class="no">InstanceMethods</span><span class="p">)</span>

    <span class="n">base</span><span class="p">.</span><span class="nf">class_eval</span> <span class="k">do</span>
      <span class="kp">alias_method</span> <span class="ss">:send_confirmation_instructions</span><span class="p">,</span> <span class="ss">:return_false</span>
    <span class="k">end</span>
  <span class="k">end</span>

  <span class="c1"># Instance methods</span>
  <span class="k">module</span> <span class="nn">InstanceMethods</span>
    <span class="k">def</span> <span class="nf">return_false</span>
      <span class="kp">false</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>What this basically does is alias the <code>send_confirmation_instructions</code> method to
a new <code>return_false</code> instance method we can inject into our model like this :</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="rouge highlight"><code data-lang="ruby"><span class="n">user</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">new</span>
<span class="n">user</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:include</span><span class="p">,</span> <span class="no">SkipConfirmationEmail</span><span class="p">)</span>

<span class="n">user</span><span class="p">.</span><span class="nf">send_confirmation_instructions</span>
<span class="c1">#&gt; false</span></code></pre>
</div>
</div>
<div class="paragraph">
<p>Therefore, my users will still have to confirm their account at first login on
the new system, but won’t be notified when I import the database a few days
before launch.</p>
</div>
<div class="paragraph">
<p>This can easily be adapted to serveral use cases, whenever you need to override
an instance method in an exisiting Ruby class, as seen <a href="https://gist.github.com/bluefuton/5851093">in this GitHub gist</a>.</p>
</div>
]]></content:encoded></item></channel></rss>