sfSyncClusterPlugin - 0.0.5

Like "symfony sync" but to a cluster

You are currently browsing
the website for symfony 1

Visit the Symfony2 website


« Back to the Plugins Home

Signin


Forgot your password?
Create an account

Tools

Stats

advanced search
Information Readme Releases Changelog Contribute
Show source | Show as Markdown

sfSyncClusterPlugin

Author

John L. Singleton

WordHustler LLC

jsingleton@wordhustler.com

http://www.wordhustler.com

Overview

The sfSyncClusterPlugin introduces a symfony sync-cluster task. The symfony sync-cluster task is intended as a replacement for the symfony sync task. While the plugin is backwards-compatible with symfony sync (they even use the same configuration files) this plugin offers several advantages over a traditional symfony sync. The symfony sync-cluster task:

  • Synchronizes a Symfony application across an unlimited number of servers. Symfony's sync only works for a single server.
  • Clears the Symfony cache on each server with a symfony cc. Doing this with sync requires an additional command.
  • Allows you to use ssh public/private keyfile authentication so you won't be prompted for a password during long deployments.
  • Allows you to deploy your web assets to a CDN without having to copy your entire site. The sfSyncCluster currently supports standard rsync transfer and transfer to Amazon S3.
  • Allows you to specify groups of servers to sync. If you have many servers, you can place them into logical groups and move them around quickly.

Installation

To install sfSyncClusterPlugin:

Listing 1 - Installing sfSyncClusterPlugin

symfony plugin-install http://plugins.symfony-project.com/sfSyncClusterPlugin
symfony cc

You can now use the symfony sync-cluster task.

Usage

The sync-cluster task behaves exactly as sync. For more information about using symfony sync, please see the Application Management section of the Symfony documentation.

To configure sfSyncCluster, you must edit your project's properties.ini file. There are three types of configurations. * A standard cluster, where everythting is synced across all machines * A rsync-based CDN, where all of the contents of the web directory are pushed to a set of CDN servers. Please note that pushing your content through a CDN requires adding rewrite rules to your Apache configuration. * A S3-based CDN, where all of the contents of the web directory are pushed to Amazon S3. Note that for this option to work you will have to have a S3 account and must place rewrite rules in your Apache configuration.

The most basic usage of sfSyncCluster the first scenario, in which you wish to deploy your entire Symfony application to a group of servers.

Listing 2 - Example of a basic sfSyncCluster configuration in myproject/config/properties.ini.

name=MyApplication
[production]
  cluster=node1.domain.com, node2.domain.com, node3.domain.com
 
  ## keyfile is optional ##
  keyfile=/Users/johndoe/keys/id_rsa
  port=22
  user=root
  dir=/var/www/html/MyApplication/

The settings you need to activate for sfSyncCluster are cluster and optionally, keyfile. The host parameter is ignored.

Note that the latest version of sfSyncCluster has support for logical server groups. To use this, you must first define the group in your properties.ini file and then reference it the cluster parameter. This offers a number of advantages (in terms of backwards compatibility) and is easy to learn. For example, using the extended syntax, we could rewrite listing 3 as follows.

Listing 3 - Example of a basic sfSyncCluster configuration in myproject/config/properties.ini, using the new logical group syntax

name=MyApplication
[production]
  cluster=applicationCluster, hotStandbys
 
 
  #logical group definition 
  applicationCluster = node1.domain.com, node2.domain.com,
  hotStandbys=node3.domain.com
 
   ## keyfile is optional ##
  keyfile=/Users/johndoe/keys/id_rsa
  port=22
  user=root
  dir=/var/www/html/MyApplication/

Running sfSyncCluster

To use this plugin, simply invoke the task from within a project.

Listing 4 - Using the sync-cluster task

/* does a dry run */
# symfony sync-cluster production

/* does it for real */
# symfony sync-cluster production go

Note that in order for sfSyncCluster to work, your servers must share the same filesystem structure for your application. For example, if dir is set to /foo/bar/myApplication/ the sync-cluster task will try to synchronize with the application at /foo/bar/myApplication/ on every server. Be warned.

Deploying your Assets to a CDN

The current release of sfClusterPlugin contains support for deploying your web assets to a rsync or Amazon S3-based CDN. When sfSyncCluster works in CDN mode, it copies the entire contents of the web directory to the CDN and excludes the rest of your application. The reason for this is two-fold: first of all, it's more secure. CDNs typically do not process dynamic pages (like PHP) and if you have any hard coded passwords (such as in databases.yml) your passwords will be exposed. Secondly, web assets have a tendency to change less frequently than their PHP counterparts, so it makes sense to be able to deploy them apart from your application.

This process typically involves 3rd party programs and complicated (read: sleep-obliterating) procedures. Fortunately, sfSyncCluster makes this a snap.

Deploying to a Rsync-Based CDN

To deploy your web assets to a rsync-based CDN, create a config/properties.ini similar to the one in listing 5.

Listing 5 - Example of a rsync-based CDN configuration in myproject/config/properties.ini, which also uses the new logical group syntax

name=MyApplication
 
 
[production-cdn]
  mode=cdn
 
  #the type parameter is optional and defaults to rsync.
  type=rsync
 
  cluster=contentServers, standbyServers, static.domain.com
 
  contentServers=content1.domain.com, content2.domain.com
  standbyServers=content3.domain.com
 
  #keyfile is optional. 
  keyfile=/Users/johndoe/keys/id_rsa
  port=22
  user=root
  dir=/tmp/web/

To deploy your web assets to this rsync CDN you would simply invoke sfSyncCluster as we did in listing 4. This time, however, be you must tell it to use the production-cdn block of the config/properties.ini file.

Listing 6 - Using the sync-cluster task for a rsync-based CDN.

/* does a dry run */
# symfony sync-cluster production-cdn

/* does it for real */
# symfony sync-cluster production-cdn go

Note that the rsync-based CDN mode is symlink-aware. All directories and plugins that live in web/ will be dereferenced and the resulting file will be copied to your CDN.

Deploying to a Amazon S3-Based CDN

To deploy your web assets to a S3-based CDN, create a config/properties.ini similar to the one in listing 7. Note that for this to work you must have an S3 account.

Listing 7 - Example of a S3-based CDN configuration in myproject/config/properties.ini, which also uses the new logical group syntax

name=MyApplication
 
[production-s3-cdn]
  mode=cdn
  type=s3
  apiKey=APIKEY
  apiSecret=40BITSECRETKEY
  s3Bucket=my-s3-cdn
 
  #in S3 lingo, this is the prefix. 
  dir=web/

To deploy your web assets to this S3 CDN you would simply invoke sfSyncCluster as we did in listing 6. This time, however, be you must tell it to use the production-s3-cdn block of the config/properties.ini file.

Listing 8 - Using the sync-cluster task for a S3-based CDN.

/* does a dry run */
# symfony sync-cluster production-s3-cdn

/* does it for real */
# symfony sync-cluster production-s3-cdn go

Note that the S3-based sync, unlike rsync, has no way of checking the modification date on the S3 CDN. For this reason sfSyncCluster will always publish your entire web directory to S3 when invoked.

Configuring Apache to Use Your Amazon S3 CDN

You should note that simply deploying your web assets to a CDN is not enough to ensure that they are served from there. To then serve files from your CDN you must either: * Hardcode links to your CDN assets (easy to do for small sites and also gives better performance than apache rewrites) * Use hardware-based CDN switching (expensive, but very very nice.) * Use an Apache rewrite rule (easy to implement for a large site, moderate performance impact from sending redirects to the CDN)

I'm going to give an example of how to use Amazon S3 as a cheap CDN through Apache rewrite rules. This technique will also work for an rsync-based CDN if you simply change the CDN urls to match yours.

Listing 9 - Example of a Apache S3-based CDN configuration in httpd.conf, which also uses the new logical group syntax

<VirtualHost *:80>

  ServerName myhost.com
  FileETag none
  DocumentRoot "/var/www/html/myapp/web"
  DirectoryIndex index.php
  Alias /sf /usr/share/pear/data/symfony/web/sf
  <Directory "/usr/share/pear/data/symfony/web/sf">
    AllowOverride All
    Allow from All
  </Directory>
  RewriteEngine On

  ##uncomment to enable debugging. 
  #RewriteLog "/var/log/httpd/rewrite_log"
  #RewriteLogLevel 9

   ## sends a 301, loads all assets in a sf* directory from amazon s3
   RewriteRule ^/sf(.*) http://s3.amazonaws.com/my-s3-bucket/web/sf$1 [R=301,L]


<Directory "/var/www/html/myapp/web">
    AllowOverride None
    Allow from All


    Options +FollowSymLinks +ExecCGI

<IfModule mod_rewrite.c>
  RewriteEngine On

  # uncomment the following line, if you are having trouble
  # getting no_script_name to work
  #RewriteBase /

  ##load images and css from S3
  RewriteRule ^images(.*)$ http://s3.amazonaws.com/my-s3-bucket/web/images$1 [R=301,L]
  RewriteRule ^css(.*)$ http://s3.amazonaws.com/my-s3-bucket/web/css$1 [R=301,L]


 # we skip all files with .something
  RewriteCond %{REQUEST_URI} \..+$
  RewriteCond %{REQUEST_URI} !\.html$
  RewriteRule .* - [L]

  # we check if the .html version is here (caching)
  RewriteRule ^$ index.html [QSA]
  RewriteRule ^([$1.html [QSA](^.]+)$)
  RewriteCond %{REQUEST_FILENAME} !-f

  # no, so we redirect to our front web controller
  RewriteRule ^(.*)$ index.php [QSA,L]
</IfModule>









  </Directory>



</VirtualHost>

License

For the full copyright and license information, please view the LICENSE file that was distributed with this source code.

Change Log

5/02/08 -- Initial Release.

5/17/08 -- Added Support for Logical Server Groups and CDNs.