sfGenerationMemcachePlugin - 1.0.0

This plugin allows you to give groups of cached data a "generation" - like a version number, which becomes part of the cache key. This number is stored under its own key in memcached and can be incremented as a way of clearing/invalidating the cache for members of this group.

You are currently browsing
the website for symfony 1

Visit the Symfony2 website


« Back to the Plugins Home

Signin


Forgot your password?
Create an account

Tools

Stats

advanced search
Information Readme Releases Changelog Contribute
Show source

sfGenerationMemcachePlugin

This plugin allows you to give groups of cached data a "generation" - like a version number, which becomes part of the cache key. This number is stored under its own key in memcached and can be incremented as a way of clearing/invalidating the cache for members of this group.

This is an alternative to symfony's built in removePattern() method, which for memcached relies on storing an array of all known cache keys in memcached (storeCacheInfo: true). This can grow very large and impact performance - prior to symfony 1.2.10 the size grows indefinitely as items are never removed.

The method used in this plugin performs significantly better than the symfony built in method on an application that maintains a cache of around 200k items.

It should be noted that this plugin doesn't allow you to be as specific when clearing cache as the built in symfony method does, as it doesn't know all of the keys. Therefore, it may not be suitable for every use case.

Requirements

The plugin will probably only help you if you are using memcached as your cache backend - for notes regarding other backends, see the appendix at the bottom.

Installation

  • Install the plugin

    symfony plugin:install sfGenerationMemcachePlugin
    
  • Edit filters.yml

    Edit your application's config/factories.yml so that your app uses the plugin's view cache manager and view cache classes:

    all:
      view_cache_manager:
        class: sfGenerationViewCacheManager
    
      view_cache:
        class: sfGenerationMemcacheCache
        param:
          host: localhost # this and the next few lines configure the memcached server - change these to suit your environment
          port: 11211
          prefix: frontend
          persistent: true
    
  • Clear cache

    symfony cc
    

Usage

For the plugin to do anything, you must configure generation_group keys in the relevant cache.yml file(s)

An example module/config/cache.yml config would be:

    all:
      enabled: on

    show:
      generation_group: profile

    index:
      generation_group: [ profile, home ] # an array can be given to apply multiple generation groups to a cache

    archive:
      lifetime: 86400

If you are configuring your caching from PHP, you can pass a generationGroup option to the ViewCacheManager's addCache method. The above examples would be:

    sfContext::getInstance()->getViewCacheManager()->addCache('module', 'show', array(
      'lifeTime' => 3600, 
      'generationGroup' => 'profile',
    );

This also accepts arrays of generation groups:

    sfContext::getInstance()->getViewCacheManager()->addCache('module', 'index', array(
      'lifeTime' => 3600, 
      'generationGroup' => array('profile', 'home'),
    );

Understanding This Config

The config in the yaml above will cause cache keys generated for module/show to be suffixed with:

/gk_profile/1

And those generated for module/index to be suffixed with:

/gk_profile/1/gk_home/1 

Those for module/archive are untouched as it has no generation_group set.

If the generation is incremented for the home generation group, the cache key for module/show remains unchanged, its cache will still be valid. The cache key for module/index will change though, it will now be suffixed with

/gk_profile/1/gk_home/2

The cache key has now changed, so the cached data will be regenerated for this action (as well as any others that specify the home generation group) - the cache has effectively been cleared for members of this generation group.

Similarly, if the generation is then incremented for the profile generation group, the cache key for module/show gets suffixed with

/gk_profile/2

and that for module/index gets suffixed with

/gk_profile/2/gk_home/2

In this case, the cache keys of both items have now changed and so both caches will be recreated.

In both of the above examples, the cache key for module/archive would be unchanged.

You may be wondering about leaving all this data in cache under old keys, and whether its a good idea. Memcached takes care of this for us: It auto expires old items from cache according to the valid times set in cache.yml. It also removes the least recently used items from memory if it runs out of free memory. This should always be the older generations of cached data. Therefore, you shouldn't run into issues as long as you have configured memcached with a sensible memory limit.

Incrementing The Generation Number For A Group

There are two ways to increment the generation for a particular generation group:

  • From the ViewCacheManager

    The sfGenerationNewCacheManager provides a method to increment the generation for a particular generation group:

    // increment the profile generation group
    $cachemanager = sfContext::getInstance()->getViewCacheManager();
    $cachemanager->incrementGenerationGroup('profile'); 
    

    There is also a wrapper method that can be passed an array of generation groups to increment the generation for.

    // increment the profile and home generation groups
    $cachemanager = sfContext::getInstance()->getViewCacheManager();
    $cachemanager->incrementGenerationGroups(array('profile', 'home'));
    
  • Using the command line task

    The plugin comes with a task to increment the generation for a given generation group:

    // increment the home generation group
    symfony cache:increment-generation home 
    

    This task takes an optional environment and application parameter.

    // increment the home generation in the dev environment for the frontend app.
    symfony cache:increment-generation --env=prod --app=frontend home 
    

    Default environment is dev, default app is frontend.

Thats it - hope someone finds it useful! Any questions, my email is on the front page, or for a quicker response, twitter me - @benlumley

Appendix 1: Notes on suitability for other backends

This plugin has been designed to be used with memcached (hence the name), however, it could be used with other cache backends:

The default file based cache should not be used with this method in my view - it does not auto-invalidate as the memory based systems do, so the cache would consume ever more disk space. The file based cache also does not suffer the performance issue this was designed to address. I believe this would also apply to SqlIte

APC, eAccelerator and Xcache all have methods to obtain a list of keys in the cache (meaning symfony doesn't store a list of them itself), so are unlikely to benefit from this method to the same degree, but developers may benefit from the functionality it offers. In this case it may be worth adapting the plugin. Creation of an increment method in a copy of the relevant sfXXXCache.class.php file from symfony/lib/vendor/cache should be all that is required, plus a change to factories.yml. The increment method is simply required to store an integer value in the cache under a passed in key, or add 1 to the existing value if it already exists. If anyone does this, contact me, and I will add you as a devloper to this plugin so that the extra functionality can be included.