2013-04-12: Speed up Magento with Memcached

Posted at 2013-04-12 16:49:25 by SHD

Magento is a highly flexible, open source e-commerce solution. Unfortunately, it can also be dog-slow, especially on servers with modest hardware specifications. Throwing extra hardware at the problem is not a great solution. There is a lot of tweaking that can be done, though. Very significant speedups may be realised through the use of Memcached. Magento supports this out of the box, but it is not enabled by default. Here's how to set that up.

Magento & Memcached

Memcached is an in-memory key-value store, which is used to access frequently used (or expensive to compute) data quickly by caching it in RAM. This way, it can be retrieved from memory without hitting the database or filesystem. As Magento can cause a pretty heavy load on a database server and uses many different XML files for its configuration and various other things, using Memcached as an extra caching layer in between can cause major speedups.

There's a few steps involved in making it work:

  1. Setting up Memcached
    1. Installing the Memcache daemon
    2. Configuring Memcached
    3. Configuring PHP
  2. Configuring Magento

1. Setting up Memcached

Before you can make use of Memcached, it needs to be installed and configured and your PHP installation needs to know it is available.

1.1. Installing the Memcache daemon

Most recent Linux distributions provide a Memcached package through their package management system. The Memcached wiki has an extensive list of instructions on how to install, depending on which Linux distribution you're running. My favourite distribution, openSUSE is missing from that list unfortunately, but instructions exist elsewhere (hint: the command there is yast2 -i memcached).

If your distribution does not provide a Memcached package, you can always build from source. The source code for the latest stable Memcached release is available directory from the home page. Extract the archive and run the standard compilation combo: ./configure
make
make install
Note: You may need to install the package libevent-dev first or build libevent from source too. Full instructions about installing from source are also available on the Memcached wiki.

When installed, you may want to set the daemon to start automatically with the rest of the system. Many Linux distributions have a simple way of doing this. If you installed from source, you should install file scripts/memcached-init from the source distribution as /etc/init.d/memcache. Then run: chkconfig --add memcached
chkconfig memcached on
service memcached start

You should now have Memcached running with its default settings. An easy way to check is by telnetting to port 11211 and giving the stats command, then quitting out of that: $ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
stats
STAT pid 32607
STAT uptime 8
STAT time 1365690137
STAT version 1.4.15
STAT libevent 1.4.11-stable
... many more lines ...
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END
quit
Connection closed by foreign host.
$

1.2 Configuring Memcached

By default, Memcached will listen on port 11211 with a memory pool of 64MB. Whether this is sufficient depends on your Magento installation's complexity and load. For my case, I could easily spare 256MB of RAM and that was more than enough to cache everything Magento needed to cache, with sufficient breathing room. Memcached usually allows anyone, anywhere to access its cache. For security, you will want to limit access. For instance, if you intend to access Memcached only from the server on which it is installed, you should have it listen only on the loopback address. If you need access from other client machines as well, you should configure the server's firewall to only allow access to TCP port 11211 from those clients' IP addresses.

You can set the memory size by changing the configuration file, which is /etc/memcached.conf from a source installation (you can create it if it doesn't exist). This is just a list of options passed as command line parameters to the memcached binary. Packaged installations might have their own way of specifying configuration. On my setup, it is a line MEMCACHED_PARAMS="-d -m 256 -l 127.0.0.1" in /etc/sysconfig/memcached. In standard format for /etc/memcached.conf, this would be only -d -m 256 -l 127.0.0.1 The -d switch tells Memcached it should be running in the background as a daemon. Then -m 256 sets it to use 256MB of memory. And finally, -l 127.0.0.1 checks for incoming connections only on the loopback interface, allowing connections only from localhost.

Note: the IP address specified here is the one on which Memcached should be listening, so the one clients are connecting to, not from i.e. it is not a source address.

More instructions are available on the Memcached wiki page for configuring a server.

1.3 Configuring PHP

Now that Memcached is available, PHP must be set up to be able to connect to it, by use of an extension. Chances are your Linux distribution already provides a pre-built package for this. See if you can find and install a package php5-memcache. If one isn't available, make sure you have PEAR installed (package php5-pear) and you can make it do the heavy lifting:pecl install memcache

You will need to add the line extension=memcache.so to your php.ini file, then Apache to load the new configuration. When Memcache support is available to PHP, something similar to the following should show up in your phpinfo(); output:

memcache

memcache supportenabled
Active persistent connections1
Version2.2.7
Revision$Revision: 327750 $
DirectiveLocal ValueMaster Value
memcache.allow_failover11
memcache.chunk_size81928192
memcache.default_port1121111211
memcache.default_timeout_ms10001000
memcache.hash_functioncrc32crc32
memcache.hash_strategystandardstandard
memcache.max_failover_attempts2020

2. Configuring Magento

When PHP can access Memcached, the next step is to make Magento actually use it. This is done by added some settings to the configuration file app/etc/local.xml, inside the <global> element:

<global> ... <cache> <backend>memcached</backend> <prefix>MAGE_</prefix> <memcached> <servers> <server> <host><![CDATA[127.0.0.1]]></host> <port><![CDATA[11211]]></port> <persistent><![CDATA[1]]></persistent> </server> </servers> <compression><![CDATA[0]]></compression> <cache_dir><![CDATA[]]></cache_dir> <hashed_directory_level><![CDATA[]]></hashed_directory_level> <hashed_directory_umask><![CDATA[]]></hashed_directory_umask> <file_name_prefix><![CDATA[]]></file_name_prefix> </memcached> </cache> ... </global>

If you run multiple Magento installations that all connect to the same Memcached server, make sure each of them has a unique value for the <prefix> setting. This ensures there won't be any conflicts with cached values from one site overwriting those for another. Of course, if you are running Memcached on a server that is not the same as the Magento webserver, be sure to adjust the IP address or hostname accordingly in the <host> setting. The same goes for changing Memcached's default port in the <port> setting.

Conclusion

It takes only a few minutes of work to set up a Memcache daemon and switch Magento to use it for caching, especially if your OS distribution already provides the appropriate packages for the Memcached server and the PHP extension. A source code installation is not overly complicated either. For my Magento installations, switching Magento to use Memcached for caching made the sites noticeably faster.

Comments

good...

Posted at 2013-04-15 12:21:00 by bhaIMAYA

Great description of how to setup memcached.

When running Magento in a load balanced cluster, if I have an instance of memcache running on each web server then list each server in <servers> section of the config will all of those memcache instances be utilized by each web server? Essentially acting as one large cache? I get the sense that this is how memcached works, but can't find a good example of it being used this way with Magento. Any experience with this?

Posted at 2013-07-19 20:32:34 by dg

Yes, dg, you can do this exactly as you describe. It would become a distributed cache. You should check whether that actually makes things faster though. The amount of data that is cached depends largely on the size of your product catalog, not so much on the amount of visitors you have. When I wrote this post, I set 256MB, but found that my shops use only between 10 and 25 MB of cache each, so I've allocated only 64MB to Memcached now. That's not really worth the overhead of having multiple servers, causing extra bandwidth use, latency, bucket selection etc. If you run a local instance only, it is probably sufficient and you could also use a more efficient socket file to connect to the instance, rather than open a TCP connection for each request. The overhead of connecting to remote instances might negate the performance benefit.

If you were to use Memcached for the user session data (which is also possible, though it requires some extra configuration), then the distributed cache would make a lot of sense. It would ensure that each of the load balanced web servers sees the same session data.

Posted at 2013-07-19 21:10:02 by Steven Don

I tried speeding up my Magento store with memcached and also with Varnish, It improved but not much as I was expecting, then I tried cloudways full page cache extension, https://www.cloudways.com/blog/free-magento-full-page-cache-cloudways/ , Its not only about cache, there is a lot matters on hosting, That's why i got fully managed magento cloud hosting, https://www.cloudways.com/en/magento-managed-cloud-hosting.php

Posted at 2017-08-18 13:19:52 by alexmorco

Post a comment

Note: HTML is not permitted, URLs will be linked automatically. Spam comments will result in a permanent ban.
Type these 4 symbols into the edit field