Blazing fast Elixir configuration

One awesome aspect of the Elixir community is its pursuit of high performance. So, I set out to benchmark and compare the two common Elixir configuration usages.

1. Module attributes

An easy way to use configuration is to use module attributes like below:

defmodule Request do
  @timeout_ms Application.get_env(:my_app, :request_timeout_ms, _default = 1000)
  def get(url) do
    HTTPoison.get(url, timeout: @timeout_ms)
    #....
  end
end

However, this approach hard-codes your configuration — Which means your configuration will need to be available at build time and will be embedded in the compiled code. So, the alternative to this is the use of `Application.get_env` during runtime. One other downside of using this is that you won’t be able to change this when the application is running (which may be fine for your use case).

2. Application.get_env

You can directly call `Application.get_env` from your code to get the configuration data like below:

timeout_ms = Application.get_env(:my_app, :request_timeout_ms, _default = 1000)
HTTPoison.get(url, timeout: @timeout_ms)

This may seem similar to the previous approach, However in this case the configuration value is read every time this code is called whereas in the case of a module attribute it is read only at compile time. Using this approach allows you to update your environment at runtime and use the updated values the next time they are read. It is by no means slow, as it is backed up by an ets table.

However, I’ve always felt a bit hesitant while using this in cases with thousands of reads per second because I expected this to be slower compared to module attrs. So, to test my hypothesis I wrote a quick benchmark.

The Benchmark

We have 2 keys in our config, `:bears` which has 3 strings and `:three_hundred_bears` which has 300 strings like below:

# config/config.exsconfig :benchmark_elixir_configuration,
  bears: ~w[ice-bear grizzly panda],
  three_hundred_bears: Enum.flat_map(1..100, fn _ -> ~w[ice-bear grizzly panda] end)

The Script

# lib/bench.exs
defmodule Config do
  @bears Application.get_env(:benchmark_elixir_configuration, :bears)
  def bears do
    @bears
  end@three_hundred_bears Application.get_env(:benchmark_elixir_configuration, :three_hundred_bears)
  def three_hundred_bears do
    @three_hundred_bears
  end
endBenchee.run(
  %{
    "noop" => fn -> for _ <- 0..1000, do: :ok end,
    "module_attribute_bears" => fn -> for _ <- 0..1000, do: Config.bears() end,
    "Application.get_env_bears" => fn ->
      for _ <- 0..1000, do: Application.get_env(:benchmark_elixir_configuration, :bears)
    end,
    "module_attribute_three_hundred_bears" => fn ->
      for _ <- 0..1000, do: Config.three_hundred_bears()
    end,
    "Application.get_env_three_hundred_bears" => fn ->
      for _ <- 0..1000,
          do: Application.get_env(:benchmark_elixir_configuration, :three_hundred_bears)
    end
  },
  time: 10,
  memory_time: 10
)

The script is a simple benchee script which runs for 10 seconds. Each function which is benchmarked reads the configuration a 1000 times to avoid
the function is too fast warning from benchee.

It contains 5 variants, here they are with their results:

1. `noop`: 24.83μs for looping a 1000 times without doing anything.
2. `module_attribute_bears`: 28.97μs for reading config using module attributes a 1000 times (which is just a function call)
3. `module_attribute_three_hundred_bears`: 29.08μs for reading a large config using module attributes a 1000 times.
4. `Application.get_env_bears`: 427.38μs for reading application config a 1000 times using `Application.get_env`
5. `Application.get_env_three_hundred_bears`: 111.10ms for reading a large application config a 1000 times using `Application.get_env`

The benchmarks show that using module attributes is 10x faster than using `Application.get_env`. However, if you look at the actual numbers using `Application.get_env` takes `0.42μs` to read a single configuration
which is pretty darn fast. So, unless you are reading configuration thousands of times you don’t have to worry about which approach you use and just use whatever is convenient.

3. Bonus — Getting the best of both worlds

If you want ability to update the environment and also want fast reads you can dynamically compile a `Config` module whenever your configuration changes.

A few takeaways

If you don’t read in from your environment a lot, it doesn’t really make a significant difference which option you choose. For performance sensitive applications that read in from the environment many times per second, then reading in the configuration from a module attribute will give your functions a noticeable boost in performance. However, with a few tweaks, writing a module to handle automatically updating environment configuration is an option for those who want the best of both worlds.

The raw results

Compiling 1 file (.ex)
Generated benchmark_elixir_configuration app
Operating System: Linux"
CPU Information: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz
Number of Available Cores: 4
Available memory: 7.49 GB
Elixir 1.6.6
Erlang 21.0Benchmark suite executing with the following configuration:
warmup: 2 s
time: 10 s
memory time: 10 s
parallel: 1
inputs: none specified
Estimated total run time: 1.83 minBenchmarking Application.get_env_bears...
Benchmarking Application.get_env_three_hundred_bears...
Benchmarking module_attribute_bears...
Benchmarking module_attribute_three_hundred_bears...
Benchmarking noop...Name                                              ips        average  deviation         median         99th %
noop                                          40.28 K       24.83 μs    ±38.58%          23 μs          52 μs
module_attribute_three_hundred_bears          34.52 K       28.97 μs    ±37.34%          28 μs          40 μs
module_attribute_bears                        34.39 K       29.08 μs    ±24.75%          29 μs          45 μs
Application.get_env_bears                      2.34 K      427.38 μs    ±30.05%         358 μs         824 μs
Application.get_env_three_hundred_bears     0.00900 K   111169.10 μs    ±23.45%      106147 μs      184344 μsComparison:
noop                                          40.28 K
module_attribute_three_hundred_bears          34.52 K - 1.17x slower
module_attribute_bears                        34.39 K - 1.17x slower
Application.get_env_bears                      2.34 K - 17.21x slower
Application.get_env_three_hundred_bears     0.00900 K - 4477.75x slowerMemory usage statistics:Name                                            average  deviation         median         99th %
noop                                           40.16 KB     ±0.00%       40.16 KB       40.16 KB
module_attribute_three_hundred_bears           37.04 KB     ±1.93%       36.88 KB       40.13 KB
module_attribute_bears                         37.04 KB     ±1.92%       36.88 KB       40.13 KB
Application.get_env_bears                     321.16 KB     ±0.00%      321.16 KB      321.16 KB
Application.get_env_three_hundred_bears     19312.34 KB     ±0.00%    19312.34 KB    19312.34 KBComparison:
noop                                           40.16 KB
module_attribute_three_hundred_bears           36.88 KB - 0.92x memory usage
module_attribute_bears                         36.88 KB - 0.92x memory usage
Application.get_env_bears                     321.16 KB - 8.00x memory usage
Application.get_env_three_hundred_bears     19312.34 KB - 480.93x memory usage

Thanks to

for helping me with this post.