Making Magento Shine with Varnish – Part 1

September 30, 2014 General, Technical, General

Developing for the web can be overwhelming – the stack of technologies involved has only grown over the years, whilst customers demand faster and more responsive websites. Performance is often an afterthought, partly because it can be tricky to define. New features are tangible and easily demonstrated, but it can be difficult to make a business case for performance during the development stage. Yet as studies by Akamai, Google and Amazon have shown, the success of e-commerce sites in particular is closely linked to how they perform. Magento is a popular e-commerce framework that offers a wealth of customisation through an extensible design, though this flexibility can easily result in slow, sluggish websites if you aren’t careful. But what does it even mean for a website to be slow or perform well?

What is performance, anyway?

Often when people talk about the performance of websites, they actually mean latency. It’s no good for business if your infrastructure can handle 10,000,000 concurrent users but every page takes 5 minutes to load. The Amazon page speed study in 2007 found that there was “a 1% decrease in sales for every 0.1s increase in response times” – and the standard is only higher these days. When a customer visits your website, there are many factors involved that determine how long it takes for the page to become usable. Some of the delay is due to the time taken by the web server to prepare and send content, which is usually referred to as the server response time. Other delays can be due to slow JavaScript or CSS that block page rendering.

The best way to get started with improving performance is to measure the page load time with an independent tool such as Google PageSpeed Insights, or Pingdom FPT. These give you a useful starting point and a good overview of where the most time is being taken. One of the most popular metrics used for analysing server-side performance is the “Time To First Byte” (TTFB), which as a concept has been measured for a long time. It’s important to note that this is not perfect since it can be confounded by internet latency and other such factors outside of your control. It can also be trivially artificially lowered whilst not impacting the total load time of the page.


If your Time To First Byte is bad, you might see this message on PageSpeed Insights

When it comes to Magento and page performance, it’s worth recognising that every page is different and even though you might optimise the load time of a product page, it doesn’t necessarily mean other pages will be any faster. It is worth running multiple pages through the online testing tools above, but generally there are five types of pages you want to benchmark to get an overview of how your site performs:

  • Homepage
  • Category
  • Product
  • Search Result
  • CMS, such as Help or FAQ

It is extremely common to find the homepage is snappy but the rest of the site is slow – the homepage tends to be where most of the effort is put in during development. Category pages are often slow because of the number of products they include, and thus the volume of database queries. Be careful of the temptation to have, for example, “show all products” since that is especially expensive for the server.


C.R.E.A.M. – Cache Rules Everything Around Magento

In an attempt to alleviate the performance problems incurred by Magento’s complex EAVĀ  design, Varien implemented several caches built on the Zend Framework. Magento supports several different caching back-ends to be specified in the local.xml configuration. Using a cache to improve performance is not a new concept by any means. Instead of doing difficult work over and over, you simply do it once and save the result. A typical Example Caching Hierarchylarge Magento deployment might have caching at all the layers illustrated in the diagram on the right. When a client makes a request from the server, you can easily visualise the various systems it passes through, from the browser’s cache all the way to the server that eventually processes it. The general rule of thumb is that caching improves performance, and caches closer to the client reduce latency further. Despite some advanced techniques it is basically impossible to rely solely on client-side caching – at some point with dynamic sites the server must become involved.

But involving the server means a massive increase in latency. Modern high performance web servers such as nginx are rarely the bottleneck when it comes to an e-commerce site though. The slowdown comes from generating dynamic pages using a language like PHP. If a request has to hit PHP at all, this adds a lot of overhead. Parsing all the code involved in a Magento store requires orders of magnitude more CPU than serving content from a cache. Even communicating over FastCGI causes some additional latency. Opcode caches such as APC or Zend help, but only so much – they are still far too slow to use with a site that will get more than a few hundreds of requests per second. Beyond these internal PHP caches, there are many caches maintained by Magento itself. Yet these still suffer from the same problem – they don’t scale, because they still involve PHP.


Varnish is an “HTTP Accelerator” – it sits in front of your web application and serves content from its cache where possible. Its main appeal is that it is fast. Really fast. However not every page can, or should, be cached. The trouble with “Full Page Caching” in the truest sense of the term is that it is useless if you are personalising content per session or per user. Elements such as the shopping cart should never be cached – otherwise you risk confusingly showing others’ carts. Likewise, if a user can create an account and login, it is important not to cache that personalised page and display it for everybody. Fortunately, there is a solution which neatly utilises the flexibility and features of both Magento and Varnish.

Magento pages are constructed out of ‘blocks’, such as the header, footer, menus, cart, lists of products and so on. Although certain content such as shopping carts or personalised greetings should never be cached, the list of products is typically going to be the same regardless of user. Turpentine is a free and open-source plugin for Magento which integrates with Varnish’s “Edge Side Includes” and allows elements of pages to be cached on the block level. This can dramatically reduce server load and also improve performance, since the most expensive parts of page rendering (the list of products, menus) are cached.

Turpentine and Varnish are certainly advanced and effective tools for taking the performance of Magento to the next level, but the decision to use them should not be taken lightly. Every Magento store is different, and depending on the plugins in use you will need to test and confirm that you have excluded the necessary blocks from the block-level caching. ESI alone can only help so much, as there can still be a delay when rendering dynamic content such as shopping carts. Ideally these dynamic elements should be loaded with AJAX so that the page can be rendered as quickly as possible.

In the following posts I will give an overview of how Varnish works, and how Turpentine integrates it with Magento. I will also demonstrate with benchmarks why this sort of caching is necessary to make Magento scale and stay fast under load.