Activerecord on MoskitoHero

An introduction to Barley, the fast model serializer

Tue, 06 May 2025 00:00:00 +0000

The story

The company I used to work for had an ageing API in v1 that was very poorly designed and slow in its implementation. It served 4 frontend platforms (a website, two mobile apps and an administration tool). Being in a startup environment, our team had to move fast, but that was at the expense of performance and maintainability. This was the situation when I arrived there. Returning JSON objects was very cumbersome. It used a custom method, available in each model that built a nested Hash, based on the attributes you gave it. That turned out to be a mess with poor performance and concerning security implications. It was difficult to maintain, with odd variable naming and behavior, so we set out to get rid of it somehow and moved to a kind of serializer DSL that was gradually introduced to replace the old mess. Later, we decided to tighten up our API with a new version to get rid of the legacy. A few requirements quickly emerged, among which:

API V2 should be easy to maintain, with clear naming conventions and namespacing
Since it is an internal API, it should be tightly coupled to the intended use and platform
API V2 should use some sort of view DSL to generate the JSON payloads
API V2 should be as predictable as possible in terms of typing and formatting.
API V2 should be super fast by default. It should be the last thing we need to optimize.

So I looked at the serializer DSL that we had designed for API v1. The first thing I did was to benchmark it against other solutions. So I stripped it out of legacy / no longer needed features, moved it to a local gem, and started playing with a few benchmarks. And it was very fast indeed ! This is the result of a benchmark I did back then:

Warming up --------------------------------------
# Omitted for conciseness
Calculating -------------------------------------
ams                    67.770  (± 1.9%) i/s -    657.000  in  10.042270s
jsonapi-rb            157.239  (± 2.3%) i/s -      1.521k in  10.010257s
barley                 96.909  (± 2.4%) i/s -    945.000  in  10.036316s
barley-cache            4.589k (± 2.8%) i/s -     44.620k in  10.010004s
ams          eager     68.237  (± 6.2%) i/s -    592.000  in  10.087602s
jsonapi-rb   eager    289.733  (± 3.7%) i/s -      2.805k in  10.033912s
barley       eager    406.732  (± 3.2%) i/s -      3.922k in  10.020076s
barley-cache eager    639.935  (± 2.4%) i/s -      6.300k in  10.053710s
with 95.0% confidence

Comparison:
barley-cache      :     4589.3 i/s
barley-cache eager:      639.9 i/s - 7.17x  (± 0.27) slower
barley       eager:      406.7 i/s - 11.29x  (± 0.49) slower
jsonapi-rb   eager:      289.7 i/s - 15.83x  (± 0.74) slower
jsonapi-rb        :      157.2 i/s - 29.17x  (± 1.08) slower
barley            :       96.9 i/s - 47.37x  (± 1.75) slower
ams          eager:       68.2 i/s - 67.25x  (± 4.69) slower
ams               :       67.8 i/s - 67.72x  (± 2.31) slower
with 95.0% confidence

Calculating -------------------------------------
# Omitted for conciseness

Comparison:
barley-cache      :      46090 allocated
barley-cache eager:     216790 allocated - 4.70x more
barley       eager:     354326 allocated - 7.69x more
jsonapi-rb   eager:     694430 allocated - 15.07x more
jsonapi-rb        :     926082 allocated - 20.09x more
ams          eager:    1068038 allocated - 23.17x more
barley            :    1102354 allocated - 23.92x more
ams               :    1299674 allocated - 28.20x more

Which is quite nice 😎.

So I made a gem out of our serializer and published it on Github. It is named Barley. I have recently made the gem even faster (from v0.9.0 on) and benchmarked it against Alba, Blueprinter, Turbostreamer, Jserializer, FastSerializer, Representable, SimpleAMS, AMS, Rails JBuilder and Panko. Here are the results of these new benchmarks:

ruby 3.4.3 (2025-04-14 revision d0b7e5b6a0) +YJIT +PRISM [x86_64-linux]
Warming up --------------------------------------
# Omitted for conciseness

Calculating -------------------------------------
                alba    401.119 (± 1.7%) i/s    (2.49 ms/i) -      2.016k in   5.027350s
alba_with_transformation
                        256.123 (± 1.2%) i/s    (3.90 ms/i) -      1.300k in   5.076497s
         alba_inline     22.749 (± 8.8%) i/s   (43.96 ms/i) -    114.000 in   5.049973s
                 ams     16.923 (± 5.9%) i/s   (59.09 ms/i) -     85.000 in   5.029061s
              barley    427.045 (± 2.6%) i/s    (2.34 ms/i) -      2.162k in   5.066130s
        barley_cache    378.285 (± 2.4%) i/s    (2.64 ms/i) -      1.927k in   5.097462s
         blueprinter    120.704 (± 1.7%) i/s    (8.28 ms/i) -    612.000 in   5.071371s
     fast_serializer    142.347 (± 1.4%) i/s    (7.03 ms/i) -    720.000 in   5.058924s
         jserializer    252.455 (± 1.6%) i/s    (3.96 ms/i) -      1.274k in   5.047584s
               panko    510.262 (± 4.7%) i/s    (1.96 ms/i) -      2.585k in   5.077381s
               rails    107.589 (± 7.4%) i/s    (9.29 ms/i) -    546.000 in   5.104715s
       representable     50.264 (± 6.0%) i/s   (19.89 ms/i) -    252.000 in   5.027987s
          simple_ams     36.584 (± 5.5%) i/s   (27.33 ms/i) -    184.000 in   5.054347s
       turbostreamer    351.551 (± 1.7%) i/s    (2.84 ms/i) -      1.776k in   5.053401s

Comparison:
               panko:      510.3 i/s
              barley:      427.0 i/s - 1.19x  slower
                alba:      401.1 i/s - 1.27x  slower
        barley_cache:      378.3 i/s - 1.35x  slower
       turbostreamer:      351.6 i/s - 1.45x  slower
alba_with_transformation:      256.1 i/s - 1.99x  slower
         jserializer:      252.5 i/s - 2.02x  slower
     fast_serializer:      142.3 i/s - 3.58x  slower
         blueprinter:      120.7 i/s - 4.23x  slower
               rails:      107.6 i/s - 4.74x  slower
       representable:       50.3 i/s - 10.15x  slower
          simple_ams:       36.6 i/s - 13.95x  slower
         alba_inline:       22.7 i/s - 22.43x  slower
                 ams:       16.9 i/s - 30.15x  slower

Calculating -------------------------------------
# Omitted for conciseness

Comparison:
               panko:     259178 allocated
              barley:     633521 allocated - 2.44x more
       turbostreamer:     641760 allocated - 2.48x more
alba_with_transformation:     818181 allocated - 3.16x more
                alba:     818241 allocated - 3.16x more
         jserializer:     822281 allocated - 3.17x more
        barley_cache:     849521 allocated - 3.28x more
     fast_serializer:    1470121 allocated - 5.67x more
         blueprinter:    2297921 allocated - 8.87x more
         alba_inline:    2736041 allocated - 10.56x more
               rails:    2757857 allocated - 10.64x more
                 ams:    4713401 allocated - 18.19x more
       representable:    5151321 allocated - 19.88x more
          simple_ams:    9017033 allocated - 34.79x more

With YJIT enabled, Barley really shines and is the first Hash serializer of the group. There are more benchmarks to be found in the Barley repository. Barley is very simple in its implementation and allocates very little. I believe that it is what makes it so fast. I’ll write another post on how I optimized it in version 0.9.0.

The Features

Barley provides:

a clear DSL to define your serializer
inline / block sub-serializer definition
russian-doll caching
type-checking with dry-types
context handling

This is how Barley works:

In the model you want to serialize

First, include the Barley::Serializable concern in your model.

# /app/models/user.rb
class User < ApplicationRecord
  include Barley::Serializable
end

Once you include this module, Barley will look for a UserSerializable class in your codebase, so you need to define one, typically in the app/serializable directory. It should inherit from Barley::Serializer

# /app/serializers/user_serializer.rb
class UserSerializer < Barley::Serializer
end

Once this is set up, the as_json method on your model will return what you have defined in your serializer class. You can also chose to use a different serializer to render your data. You can use the serializer method in your model. This method also allows you to pass a cache argument.

# /app/models/user.rb
class User < ApplicationRecord
  include Barley::Serializable

  serializer MyCustomSerializer, cache: true
  # or
  serializer MyCustomSerializer, cache: {expires_in: 1.hour}
end

Another way to achieve this is through the as_json method:

User.first.as_json(serializer: MyCustomSerializer, cache: {expires_in: 1.hour})

Please note that Barley overrides the as_json method and does not provide it with standard only, include or except arguments. This is because the purpose of this serializer is precisely to do that job. The root argument works, though.

Defining the serializer

Let’s go back to the serializer class. Barley provides a simple DSL that allows you to define your JSON in a readable and maintainable way. Here is an example definition that covers most use cases:

# /app/serializers/user_serializer.rb
class UserSerializer < Barley::Serializer

  attributes id: Types::Strict::Integer, :name

  attribute :email
  attribute :value, type: Types::Coercible::Integer

  many :posts

  one :group, serializer: CustomGroupSerializer

  many :related_users, key: :friends, cache: true

  one :profile, cache: { expires_in: 1.day } do
    attributes :avatar, :social_url

    attribute :badges do
      object.badges.map(&:display_name)
    end
  end

end

For type-checking to work, you need to define a Types module like this, somewhere in your codebase:

module Types
  include Dry.Types()
end

1. Attributes

They can be defined as an array with the plural attributes method. It can be an array of symbols, or an array of hashes in the key: TypeDefinition pattern, or a mix of both. They can also be defined with a singular attribute method, that accepts a type keyword argument. If type is included in an attribute’s definition, it will raise an error. You can use types to coerce types and define constraints. Please refer to the dry-types gem for more options. If you give a block to the attribute method, you can also return anything you like. You have the object object at hand that refers to the serialized model.

2. Associations

They are defined with the one or many methods. It will use the default serializer if no argument is given. You can provide it with a serializer and a cache option. These methods do not provide a type argument, as it makes no sense to do so. You can also decide to give it a block, that allows you to define the association’s serializer inline.

Wrapping it up

The Barley gem’s goal is to provide a simple DSL to define your JSON payloads with a type-proof option, and render it in a very fast and efficient manner. It is probably not as feature-packed as other serializers available to the rails community, but it is a no-brainer, yet powerful option that you should consider if you are looking for performance and simplicity. I’d be happy to see more projects starting to adopt it and would love the community to provide optimizations to make it even better.

Track and debug your ActiveRecord SQL queries with annotations

Tue, 10 Dec 2024 00:00:00 +0000

I recently discovered a handy feature introduced in Rails 6. This feature allows you to annotate your SQL queries with comments directly embedded in the query itself. I know, I’m a bit late to the party, but I suspect I’m not the only one, so here’s a quick rundown.

In our app, we have a deeply nested hash called public_data that’s often used to render a record’s JSON payload, along with its associations, in the controller.

This public_data hash is built from a mix of attributes and model methods. However, it also calls the public_data methods of associated models, which might, in turn, call back to the original model’s public_data. This circular complexity has been a challenge, and while we’ve started transitioning to the Barley serializer gem for better structure, we still rely on this method in parts of our codebase for backward compatibility.

So when @user.public_data is called in the controller, it often triggers a long chain of database queries. Tracking down where a specific relation is being called—whether in User, Profile, or Subscription--can be daunting.

While Rails logs are helpful, they require sifting through the output to correlate the log entry with the SQL query. This becomes even trickier when debugging in remote environments and searching through extensive logs in tools like New Relic or CloudWatch.

Enter ActiveRecord::Relation#annotate…

This simple method lets you add comments to your SQL queries, embedding contextual information directly within the query.

Here’s how it works:

users = User.active.where(score: ..5).annotate("called from Admin::UsersController")

#> SELECT * FROM `users` WHERE `last_visit_at` >= '2024-11-10 16:53:05'
#  AND `score` <= 5 /* called from Admin::UsersController */ ORDER BY `id` ASC

This provides a clear indication of where the query originated. You can also include additional context, such as the current locale, user, or order ID, depending on your needs.

For example, adding an annotation inside the public_data method on my Subscription model makes it much easier to trace where the query originated in the chain.

Another great feature is the ability to stack annotations. Let’s say we add an annotation in the active scope of the User model:

class User < ApplicationRecord
  scope :active, -> { where(last_visit_at: 1.week.ago..).annotate("user active in the past week") }
end

Calling the same query as before would generate:

users = User.active.where(score: ..5).annotate("called from Admin::UsersController")

#> SELECT * FROM `users` WHERE `last_visit_at` >= '2024-11-10 16:53:05'
#  AND `score` <= 5 /* user active in the past week */ /* called from Admin::UsersController */
#  ORDER BY `id` ASC

Neat, right? This approach brings more clarity and structure to your logs, making them easier to query and group in log viewers. Additionally, these annotations appear in your database logs, offering a seamless way to correlate them with application logs.

Possible applications include:

Tracking queries in background jobs
Tagging controller context, headers, or anything else inside the queries
Debugging N+1 or slow queries
Expliciting the scope of queries
And so much more creative stuff!

That said, annotations come with a slight tradeoff in query readability. It’s essential to be deliberate about what and when to log, and to periodically review whether the annotations remain relevant and valuable.

Even if used exclusively for local debugging, SQL annotations are an excellent addition to your toolbox.