|
|
ajDoctrineLuceneablePlugin - 0.1.1This plugins allows for easy lucene full text integration in doctrine and the admin generator |
|
 |
9
users
Sign-in
to change your status |
This plugins allows for easy lucene full text integration in doctrine and the admin generator |
This plugin allows for convenient and easy doctrine model integration with Zend Lucene, through actAs: [Lucenable].
| Name | Status | |
|---|---|---|
A. van Waart |
lead | ln.sdd <<ta>> wvdnera |
| Version | License | API | Released |
|---|---|---|---|
| 0.1.1beta | MIT license | 0.1.0beta | 05/10/2010 |
| Version | License | API | Released |
|---|---|---|---|
| 0.1.1beta | MIT license | 0.1.0beta | 05/10/2010 |
Array
Array
This is the documentation of the Doctrine Lucene Behaviour. This will provide you with full text search in the admin generator filters, and a convenient way to roll out your own powerful search engine.
Knowledge of lucene is not nescesairy, but it is recommended. The jobeet tutorial on searching is an excellent place to start.
This tutorial will show you how to get fulltext doctrine query with just a few lines.
1. Install the plugin
symfony plugin:install ajDoctrineLucenablePlugin --stability=beta
2. Configure the plugin
By default, no configuration is required, but it's good to look into the lucene.yml file. To configure the plugin: copy the lucene.yml from the ajDoctrineLucenablePlugin/config/lucene.yml to your SF_ROOT/config. See the yml file for details on the configuration
2. Enable the lucene behaviour for your model
Project:
columns:
id:
type: integer(4)
primary: true
autoincrement: true
location:
type: string(255)
notblank: true
body:
type: clob
name:
type: string(255)
notblank: true
actAs:
Luceneable:
fields:
- name: unstored
- location: unstored
- body: unstored
This is a key/value pair, in which the name corresponds with the field (actually the model method for the field, so it can be your custom getFullName() method, then you use the name 'full_name'), and the value corresponds with the Zend Lucene field type (for reference http://framework.zend.com/manual/en/zend.search.lucene.overview.html). This can either be keyword, unindexed, text or unstored. If you're unsure, you will probably want to use unindexed.
Since we 99% of the time only use the lucene index to retrieve document id's, the default value is UnStored. This saves the metadata, but not the data.
3. (re)build the model
symfony doctrine:build --model
4. Add the filter to the model
To create a searchable filter for a model, add this to the ProjectFormFilter.class.php:
class ProjectFormFilter extends BaseProjectFormFilter { public function configure() { $this->widgetSchema['search'] = new sfWidgetFormFilterInput(array('with_empty' => false)); $this->validatorSchema['search'] = new sfValidatorPass(array('required' => false)); $this->useFields(array('search')); } public function addSearchColumnQuery($query,$field,$value) { $ids = LuceneSearch::find($value['text']) ->fuzzy() // search fuzzy? ->in($this->getModelName()) ->setFilterQuery($query); return $query; } }
5. You're done.
Project:
columns:
id:
type: integer(4)
primary: true
autoincrement: true
location:
type: string(255)
notblank: true
body:
type: clob
name:
type: string(255)
notblank: true
actAs: [Luceneable]
In lib/models/doctrine/Project.class.php: Here we add to code that is being called when the model is inserted or updated. This example shows how to load and parse a HTML document to the lucene index. The updateLucene method should always return a Zend_Search_Lucene_Document.
public function updateLucene() { $doc = Zend_Search_Lucene_Document_Html::loadHTML($this->getBody()); $doc->addField(Zend_Search_Lucene_Field::Unstored('name', $this->getLocation(), 'utf-8')); $doc->addField(Zend_Search_Lucene_Field::Unstored('location', $this->getLocation(), 'utf-8')); return $doc; }
You can override the parameters from schema.yml from the model by defining a public attribute $luceneSearchFields.
public $luceneSearchFields = array( 'name' => 'unstored', 'location' => 'text' );
To create a searchable filter for a model (and the admin generator), add this to the lib/filters/doctrine/FormFilter.class.php:
class OrganisationFormFilter extends BaseOrganisationFormFilter { public function configure() { $this->widgetSchema['search'] = new sfWidgetFormFilterInput(array('with_empty' => false)); $this->validatorSchema['search'] = new sfValidatorPass(array('required' => false)); $this->useFields(array('search')); } public function addSearchColumnQuery($query,$field,$value) { LuceneSearch::find($value['text']) ->fuzzy() // do we want to search fuzzy? ->in($this->getModelName()) ->setFilterQuery($query); return $query; } }
Adding the ->fuzzy() true will mean fuzzy searching (or in fact adding a ~ behind every keyword in the search string.) This means fuzzy searching. This is great for newbees and noobs, but not so great if you wish to use the cool Zend Lucene Query Language.
The trick in creating a simple search engine is that for each model you have to render a custom partial for that model.
in search/actions/actions.class.php:
public function executeSearch(sfWebRequest $request) { $this->getResponse()->setTitle('Search results for: ' . $request->getParameter('query')); $this->searchquery = $request->getParameter('query'); $query = LuceneSearch::find($query)->fuzzy()->in(array('News','Member','Project','Event','Content','Organisation')); $this->results = $query->getRecords(); $this->hits = $query->getHits(); }
in search/templates/searchSuccess.php:
<?php $count = count($hits); if ($count == 0) { echo "No results found for '<strong>" . $searchquery . "</strong>':"; } elseif ($count == 1) { echo "1 result found for '<strong>" . $searchquery . "</strong>':"; } else { echo sprintf("%d results found",$count) . " for '<strong>" . $searchquery . "</strong>':"; } ?> <?php foreach ($hits as $hit) { echo get_partial($hit->model, array( 'obj' => $results[$hit->model][$hit->pk], 'pk' => $hit) ); } ?>
in the get_partial() line lies the magic, here you can load for each model a seperate template. My template for news looks like this:
in templates/_News.php
<?php use_helper('Date'); ?> <?php $data = $obj; if ($data->is_published): ?> <div class="contentItem clickableItem"> <div class="itemDescription"> <p> <h1 class="searchCatagory">News</h1><br /> <?php echo format_date($data['created_at'], 'EEE dd MMMM yyyy') ?>: <h1><?php echo $data['title'] ?></h1> <?php $abstract = substr($data['abstract'], 0, 180); echo nl2br($abstract) . '... <br />' . link_to('> READ MORE','news/show?id='. $data['id'],array('class'=>'follow')); ?> </p> </div> <div class="itemImage"><img src="/uploads/news/s/<?php echo $data['image']; ?>"></div> <div class="borderDiv"> </div> </div> <?php endif; ?>
Likewise, you can create such a template for each model that you wish to include in your search engine. (in this case for: 'News','Member','Project','Event','Content','Organisation')
There is a convenient class to search models:
$query = LuceneSearch::find($string) ->in('model1','model2','model3');
This method will return the Pk's (primary keys, per model)
$hits = $query->getHits(); // get an array of lucene hits $ids = $query->getPks(); // get an array of lucene hits in the form: array ( 'Model' => array(1,2,3,45,5,6,7), 'Model2' => array(5,2,1,4) )
$ids = LuceneSearch::find($value['text']) //->fuzzy() ->in($this->getModelName()) ->getPksForModel(); $query = Doctrine_Query::create() ->select('m.*') ->from('myModel') ->andWhereIn('id',$ids);
This will return you the data from the database.
This plugin provides a task to (re)index your models, when data has been modified in the database by other means that doctrine.
usage
symfony lucene:reindex
This will use the models defined in lucene.yml (see example in the ajDoctrineLucenablePlugin/config) folder for documentation
symfony lucene:reindex Model1 Model2 This will update and optimize Model1 and Model2.
usage
symfony lucene:optimize
This will optimize the index file.
By default, the plugin will use the packaged Zend Framework. To disable this behaviour:
in: lucene.yml
use_packaged_lucene: false
Then add in: ProjectConfiguration.class.php:
public function setup() { // ... load plugins here ... // // Here add the listener for the event lucenable.autoload, and connect it to your own // ZF autloading method $this->dispatcher->connect('luceneable.autoload','ProjectConfiguration::registerZend'); } static protected $zendLoaded = false; static public function registerZend() { if (self::$zendLoaded) { return; } set_include_path(sfConfig::get('sf_lib_dir').'/vendor'.PATH_SEPARATOR.get_include_path()); require_once sfConfig::get('sf_lib_dir').'/vendor/Zend/Loader/Autoloader.php'; Zend_Loader_Autoloader::getInstance(); self::$zendLoaded = true; }
Download Zend library from: http://framework.zend.com/download/current/
To use a minimal zend library, download the entire zip, and from the ZendFramwork-1.x.x/library/Zend copy only these to your /lib/vendor/Zend folder:
Zend/Loader/* Zend/Search/* Zend/Exception.php Zend/Loader.php
The plugin consists of 5 classes * LuceneHandler (Global class, to handle the autoloading and singleton class for lucene) * LuceneRecord (Callback methods, for the lucenable listener) * LuceneSearch (to ease the searching of indexes) * Luceneable (doctrine Behvaiour) * LuceneableListener (Doctrine_Record_Listener)
