While I am obviously biased towards Elixir and the role it plays in the performance of web applications, I will do my best to explore fallacies that overplay and underplay the role of performance in web applications. I will also focus exclusively on the server-side of things (which, in many cases, is a fallacy in itself).

Fallacy 1: Performance is only a production concern

In my opinion, the most worrisome aspect of performance discussions is that they tend to focus exclusively on production numbers. However, performance drastically affects development and can have a large impact on developers. The most obvious examples I give in my presentations are compilation times and/or application boot times: an application that takes 2 seconds to boot compared to one that takes 10 seconds has very different effects on the developer experience.

Even response times have direct impact on developers. Imagine web application A takes 10ms on average per request. Web application B takes 50ms. If you have 100 tests that exercise your application, which is not a large number by any measure, the test suite in one application will take 1s, the other will take 5s. Add more tests and you can easily see how this difference grows. A slow feedback cycle during development hurts your team’s productivity and affects their morale. With Elixir and Phoenix, it is common to get sub-millisecond response times and the benefits are noticeable.

When discussing performance, it is also worth talking about concurrency. Everything you do in your computer should be using all cores. Booting your application, compiling code, fetching dependencies, running tests, etc. Even your wrist watch has 2 cores. Concurrency is no longer the special case.

However, you don’t even need multiple cores to start reaping the benefits of concurrency. Imagine that in the test suite above, 30% of the test time is spent on the database. While one test is waiting on the database, another test should be running. There is no reason to block your test suite while a single test waits on the database and this change in itself already improves build times.

If multiple cores are available, you should demand even more gains in terms of performance throughout your development and test experiences. The Elixir compiler and built-in tools will use multiple cores whenever possible. The next time a library, tool or framework is taking too long to do something, ask how many cores it is using and what you can do about it.

Fallacy 2: Threads are enough for multi-core concurrency

Once we start to venture into concurrency, a common fallacy is that “if a programming language has threads, it will be equally good at concurrency as any other language”. To understand why this is not true, let’s look at Amdahl’s law.

To quote Wikipedia, Amdahl’s law is a formula which gives the theoretical speedup in latency of the execution of a task at fixed workload that can be expected of a system whose resources are improved:

Amdahl’s law applied to number of processors. From Wikipedia, CC BY-SA 3.0.

The graph above shows that the speedup of a program is limited by its serial part. If only 50% of the software is parallelizable, the theoretical maximum speedup is 2 times, regardless of how many cores you have in your system.

If 50% of your software is parallelizable, going from 4 to 8 cores gives you only a 11% speed up. If 75% of the software is parallelizable, going from 4 to 8 cores gives you a 27% increase.

In other words, threads are not enough for most web application developers if they still have to explicitly reach out for them. Instead we need abstractions that are used as building blocks. We need good programming models, efficient data structures, and tools. If only a limited part of the software is parallelizable, you will be quickly constrained by Amdahl’s law. Threads are necessary but not sufficient. Writing maintainable and effective concurrent software takes much more.

Fallacy 3: Conclusions drawn from average response times

Another common fallacy in such discussions is when conclusions are drawn based on average data: “Company X handles Y req/second with an average of Zms, therefore you should be fine”.

Here is why conclusions on this data is not enough. First of all, most page loads will experience the 99% server response (also see Everything you know about latency is wrong for more discussion). Whenever you measure averages, also measure the 90%, 95% and 99% percentiles.

Furthermore, in our experience, clients rarely have performance issues during average load, but rather when there are spikes in traffic. It is easy to plan for your average load. The challenge is in measuring how your system behaves when there is a surge in access. When discussing and comparing response times, also ask for the high percentiles, delays and error rates in case of overloads.

Finally, the server response time as a metric is inherently limited. For instance, a fast server means nothing if the client-side is a mess and takes seconds to load. Instead of measuring a single request, consider also measuring how the user interacts with the website within certain goals. Let’s see an example.

Imagine that your application requires the user to confirm their account in order to access part of its functionality (or all of it). Now, preparing for a spike in traffic, you cached your home page as well as your sign-up form. Requests start to pour in and you can see your website is responding fairly well, with low averages and even low 95% percentiles. You consider it a success.

The next day, you are measuring how users interacted with your application and you could notice a unusually high bounce rate when the servers were on high load. Further analysis reveal that, even though the response times were excellent, the messaging system was clogged and instead of waiting 30 seconds to receive a message with instructions to confirm their account, users had to wait 10 minutes. It is safe to say many of those users left and never came back.

Response times are not enough. For queues/jobs, you want to at least measure arrival rates, departure rates and sojourn time. For this particular sign-up feature, you should measure the user engagement: from signing up, to scheduling the message, to delivering the message and the final user interaction with it.

Fallacy 4: Cost-free solutions

This is probably the most common fallacy of all.

If you complain a certain library or framework takes a long time to boot, someone may quickly point out that there is a tool that solves the booting problem by having a runtime always running on the background.

If your web application takes long to render certain views, you will be told to cache it.

The trouble is that those solutions are not cost-free and their cost are often left unsaid. When the tool that runs your application in the background fails, your developers will be the ones debugging it. Between having a solution that addresses a certain problem and not having the problem at all, I prefer the second.

Sometimes the lack of performance or proper tools will affect how your team designs and implements a feature. Time spent on caching and cache expiration is time that could be spent developing features. It is often joked that “cache invalidation and naming things are the two hard things in computer science”. We have discussed similar trade-offs in a previous article about stateful and stateless applications – where a performant solution leads to benefits from development to deployment.

This fallacy also happens when arguing in favor of technologies that are seen as performance centric. For example, if you want to use Elixir or Go, you will have to learn the underlying abstractions for concurrency, namely processes and goroutines, which is a time investment. If you want your tests to run concurrently when talking to the database in your Phoenix applications, you need to learn the pros, cons and pitfalls of doing so, a topic we covered in depth in the “What’s new in Ecto 2.0” free ebook.

It is important to make those costs explicit and part of the discussions.

Fallacy 5: Performance is all that matters

For the majority of companies and teams, that’s simply not the case. Therefore, if you are planning to move to another technology exclusively because of performance, you should have numbers that back up your decision.

Similarly, we often see new languages being dismissed exclusively as “performance fallbacks”, while in many of those languages performance is typically a side-effect. For example, Elixir builds on the Erlang VM and focuses on developer productivity and code maintenance – and that’s why all of us at Plataformatec are proud of it. If you can get extra performance in production from it, that’s a nice bonus.

At the end of the day, the discussion about performance is quite nuanced. It is important to know what to measure and how to interpret the data collected. We have learned that performance and concurrency models matter way beyond your production environment and have a large impact in development and testing. And there are no cost-free solutions, be it adding and maintaining a caching layer or picking up a new programming language.

The post The fallacies of web application performance first appeared on Plataformatec Blog.

1. Reduce Devise.stretches

Add the following to your spec/test helper:

Devise.stretches = 1

Explanation: Devise uses bcrypt-ruby by default to encrypt your password. Bcrypt is one of the best choices for such job because, different from other hash libraries like MD5, SHA1, SHA2, it was designed to be slow. So if someone steals your database it will take a long time for them to crack each password in it.

That said, it is expected that Devise will also be slow during tests as many tests are generating and comparing passwords. For this reason, a very easy way to improve your test suite performance is to reduce the value in Devise.stretches, which represents the cost taken while generating a password with bcrypt. This will make your passwords less secure, but that is ok as long as it applies only to the test environment.

Latest Devise versions already set stretches to one on test environments in your initializer, but if you have an older application, this will yield a nice improvement!

2. Increase your log level

Add the following to your spec/test helper:

Rails.logger.level = 4

Explanation: Rails by default logs everything that is happening in your test environment to “log/test.log”. By increasing the logger level, you will be able to reduce the IO during your tests. The only downside of this approach is that, if a test is failing, you won’t have anything logged. In such cases, just comment the configuration option above and run your tests again.

3. Use shared connection with transactional fixtures

If you are using Capybara for javascript tests and Active Record, add the lines below to your spec/test helper and be sure you are running with transactional fixtures equals to true:

class ActiveRecord::Base
  mattr_accessor :shared_connection
  @@shared_connection = nil

  def self.connection
    @@shared_connection || retrieve_connection
  end
end

# Forces all threads to share the same connection. This works on
# Capybara because it starts the web server in a thread.
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection

Explanation: A long time ago, when Rails was still in 1.x branch, a new configuration option called use_transactional_fixtures was added to Rails. This feature is very simple: before each test Active Record will issue a begin transaction statement and issue a rollback after the test is executed. This is awesome because Active Record will ensure that no data will be left in our database by simply using transactions, which is really, really fast.

However, this approach may not work in all cases. Active Record connection pool works by creating a new connection to the database for each thread. And, by default, database connections do not share transactions state. This means that, if you create data inside a transaction in a thread (which has its own database connection), another thread cannot see the data created at all! This is usually not an issue, unless if you are using Capybara with Javascript tests.

When using Capybara with javascript tests, Capybara starts your Rails application inside a thread so the underlying browser (Selenium, Webkit, Celerity, etc) can access it. Since the test suite and the server are running in different threads, if our test suite is running inside a transaction, all the data created inside the test suite will no longer be available in the server. Alternatively, since the server is outside the transaction, data created by the server won’t be cleaned up. For this reason, many people turn off use_transactional_fixtures and use Database Cleaner to clean up their database after each test. However, this affects your test suite performance badly.

The patch above, however, provides a very simple solution to both problems. It forces Active Record to share the same connection between all threads. This is not a problem in your test suite because when the test thread is running, there is no request in the server thread. When the server thread is running, the test thread is waiting for a response from the server. So it is unlikely that both will use the connection at the same time. Therefore, with the patch above, you no longer need to use Database Cleaner (unless you are using another database like Mongo) and, more importantly, you must turn use_transactional_fixtures back to true, which will create a transaction wrapping both your test and server data, providing a great boost in your test suite performance.

Finally, if any part of your code is using threads to access the database and you need to test it, you can just set ActiveRecord::Base.shared_connection = nil during that specific test and everything should work great!

Conclusion

That’s it! I hope you have enjoyed those tips and, if they helped you boost your test suite performance, please let us know in the comments the time your test suite took to run before and after those changes! Also, please share any tips you may have as well!

The post Three tips to improve the performance of your test suite first appeared on Plataformatec Blog.

First, what is a backend?

I18n.t, I18n.translate, I18n.l and I18n.localize methods are actually just wrappers to I18n.backend, which is actually who does all the heavy lifting. This means that you can change your backend to other stuff, as long as it respects the required API.

By default, I18n comes with the Simple backend, but others are available. For example, I18n has an ActiveRecord, which stores translations in the database. This is useful in cases someone needs to change translations through a web interface. To use it, you just need to do:

  I18n.backend = I18n::Backend::ActiveRecord

There are a couple other backends, like a backend which implements fallbacks, so if something cannot be found in a specified language, like german (:de), it can fallback to english (:en). You can check the whole list, but for this while, we are going to focus on the two new backends extensions.

Fast

Fast, means fast. And oh, boy, this one is fast. This extension flattens translations to speed up the look up. For example, the following hash { :a => { :b => { :c => :foo } } }, gets flattened to { :"a.b.c" => "foo" }, so instead of recursively looking into hashes, it looks just once. The obvious expense is that whenever you are storing translations, you need to flatten the translation hash, and it takes more time than usual.

In order to measure different backend implementations, I pushed some benchmark setup to the I18n repository. The current setup measures the storage time, the time it takes to translate a key (the depth of the key means how many nested hashes it takes), the time to translate a key falling back to the default key and the time translate a key (at depth 5) and interpolate. The results comparing the Simple backend without and with Fast extension are showed below:

In other words, a simple lookup using the Fast extension is 3 to 4 times faster than the Simple one. Besides, configuring your application to use it is very simple:

  I18n::Backend::Simple.send :include, I18n::Backend::Fast

Nice!

Interpolation compiler

The InterpolationCompiler is a backend extension which extracts all required interpolation keys from a string, leaving just the minimum required to runtime. Imagine the following string: "This is a custom blank message for {{model}}: {{attribute}}". This extension annotates the string so it knows that it needs to interpolate both model and attribute, and points exactly where the interpolation should happen. We can compare the Simple backend without and with the InterpolationCompiler below:

The InterpolationCompiler just changes the time taken when we have interpolation keys, without affecting too much the other translations. You can add it to your app as easy as the Fast backend:

  I18n::Backend::Simple.send :include, I18n::Backend::InterpolationCompiler

Run, I18n, run!

But the best is still coming! Fast and InterpolationCompiler can actually be used together, to achieve unseen performance in I18n. The benchmark speaks for itself:

While we speed up the performance in around four times in simple lookups, Fast and InterpolationCompiler improvements get combined whenever we need to use interpolation, becoming around six times faster!

As said previously, both extensions increase the time taken to store translations as side-effect. Such can be viewed below:

The yaml hash used in for the benchmark, is relatively slow, but it shows how the time taken to store translations grows with such extensions. But remember, you are constantly storing translations only in development (before each request is processed). In production, the translations are stored at startup and that is it.

Using within Rails

You should be able to use such features today in Rails 2.3.5 and it will also be possible in Rails 3. You just need to install the I18n gem and configure it in your environment.

Why care?

All the times shown are in miliseconds. In other words, why care? If you are building a simple application, using just one language, those improvements likely won’t change anything. But in an application which relies on I18n, during a request/response lifecycle, I18n is invoked many times: error messages for models, flash messages, page titles, e-mail subjects, page content, date and time localization, pluralization rules and even in many of ActionView helpers. So in such cases, it’s worth to give such extensions a try.

Running benchmarks on your own

If you want to run benchmarks on your own, it’s quite simple. You just need to do:

git clone git://github.com/svenfuchs/i18n.git
cd i18n
ruby benchmark/run.rb

Credits

The possibility to have backends and such extensions is due to Sven Fuchs, which leads the I18n group quite well.

Many of the backends were added by I18n community, and the Fast and InterpolationCompiler were created by thedarkone.

Guys, I owe you a beer!

Enjoy!

The post Run, I18n, run! first appeared on Plataformatec Blog.