Achieving sub 5ms performance for authenticated users with Drupal on shared hosting

Disclaimer: This is a work in progress, and as such probably not *quite* suitable for production sites unless you understand the issues and want to "chip in". However the instructions are pretty simple and will work on most hosting set ups from a modest shared hosting account to a server farm.

We all know the problem: Drupal has an excellent caching system that serves pages extremely quickly to *anonymous* users. Unfortunately, for authenticated (ie logged in) users, caching becomes a *much* more complicated problem. The current OOTB solution to this complication is to simply disable caching for authenticated users.

The situation is pretty dire. With stock Drupal a page can be served to an anonymous user in around 30ms, for an authenticated user on a site with a reasonable selection of modules, this time can easily increase to 650ms or even 900ms. Worse, because an authenticated user page request uses so much more system resource, it's much more difficult to scale sites to large numbers of authenticated users. Even a  fairly beefy dedicated server can probably only manage somewhere between 2 and 10 authenticated requests per second and as the number of concurrent requests increases, the server quickly gets swamped and page generation time increases to between 5 and 10 seconds per page!!

Fortunately, there is a solution ! After much investigation I discovered the authcache module. This looked to be close to the solution we're all looking for but there was no D7 version and didn't seem to be any progress from the module maintainer in upgrading.

What you need
If you'd like to see your Drupal site *flying* along with a <5ms page generation time, this is what you're going to need:

• Drupal 7 or Pressflow - I'm using Pressflow but this approach will achieve comparable results with stock Drupal.
• Authcache module.
• (Memcached module no longer needed :) )
• A small core hack - I'm sure there's a way round needing this but I haven't found it yet
• An understanding of the caveats

General Instructions

1. Install authcache module - carefully follow the fairly straightforward instructions in the README.txt

2. At this point you should find authcache is working and your page generation times (for cached pages) are very much faster (in the area of 10ms for me). If not, it may help to run "clear cache" and also log out and log back in again. Also note that caching is still disabled for the "Admin" user (ie uid=1) so you won't notice any improvement whilst running as "Admin".

Caveats

1. Whilst providing very fast response times in many circumstances, this solution is not a "magic bullet". Very fast response times are possible under all sorts of "personalisation use cases" but it requires work and understanding. Sometimes *much* work and understanding.

2. The authcache module works by generating a hash of all the groups a user belongs to and using that hash as a prefix to the "cache key". The effect of this is that a user with an obscure or unique combination of groups will be much less likely to get a "cache hit" and will therefore get a slow (ie 650ms) page generated from scratch.

3. This is currently a work in progress. At some point (soon) I should be able to make the Authcache module a "point and click", zero configuration install, but for now, you early adopters will have little work and understanding to do. Any site owners out there with high traffic authenticated user Drupal sites that would like to help me "remain alive" whilst working on this project would be very welcome to get in touch.

4. If you edit a node / page you'll notice you need to clear the cache to publish the changes. I'll solve this problem (easily) but there are likely to be other similar issues waiting to be discovered.

5. The authcache module uses javascript to personalise the page for individual users (ie "hello, user name" and all that). You should read the authcache documentation fully to understand the potential implications on how you should develop your site.

6. one issue that has become apparent in porting the module is that there is currently a "security consideration". Authcache potentially makes all cached content available to anyone who has the appropriate cache_key cookie set. This makes it somewhat easy for a "curious" person to fake the appropriate cookie value and gain access to "private" content from the cache.

This means that authcache is currently only suitable if you want to speed up sites with roles used for "personalising" the content. Authcache currently only provides very "casual" protection to private information so if you want to keep certain content totally protected you will need to wait for a subsequent release of this module.