History of SearchPackage

Comparing versions
Version 4Current version
This package makes any content on your site searchable.

Well - not ANY content. Currently, wiki pages, blog headers and entries, articles and comments.

Random Indexing

The indexing engine in Bitweaver would, every so-many clicks, would pick a random piece of content and index it. (This number of clicks is the "Search Refresh Rate" in administration->search->search settings). This tactic, I presume, is designed to limit index table additions, and prevent the overloading of the server or database engine.

The issue with "random" indexing is that when you submit an article, or wiki page - that page wouldn't necessarily ever get in the index. It is based on chance. If your site had enough traffic, the page would eventually get picked up. For smaller sites, however, this technique fails to deliver expected results - namely, submitted contet should get indexed and be findable using search.

Index On Submit

There is now In CVS (after the 1.2.1 release) an additional indexing option called "Index On Submit". This causes content to be indexed immediately after it is created (except articles that require approval. That content is indexed only after approval).

Indexing Functions

You can now delete or rebuild indexes for the whole site - or just a particular type of tiki content. you can also clear out the searchwords and syllables (words and partial words entered into search).

Remaining Issues

Remaining issues that need to be tackled:
  • Fix pagination of results
    • Once this is complete - it will be recommended that it is run an any new install of this version of search.
  • Fix the search filters (currently broken)
  • Allow Stop Words
  • Allow search to search phpbb
  • Make the search cache work
  • Make search into a Bitweaver "service"


 

Overview

This package makes all liberty content on your site searchable. Currently this includes articles, wiki pages, blog headers, blog posts and comments. Search is a service and automatically indexes content when the content is saved. The exception is articles which are only indexed after they are approved.

Developers: check SearchPackageDevNotes

Search Settings

Administration -> Search -> Search Settings

Search Statistics

Record searches made and their frequency. Checking this makes "search statistics" appear on the search admin menu, and gives you access to the statistics page.

Minimum number of letters in search word

Index words that are this number of letters or longer only. I set mine to 4, then words like "the" aren't indexed.

Maximum number of words

This applies to partial word searches. It is the maximum number of words containing a "partial word" that can be used in any one search. So, if you type in "three" - search will pick up three and twentythree. That is two - the limit is 100 by default.

Age in hours of Search cache

Again - applies to partial word searches. It defines the maximum age of cached search results for any given partial word. The results cache will be used to provide a search result if it is available, and will be cleared after either the age, or when the results cache reaches it's limit (LRU List Length)

LRU Purging Rate

Applies to partial word searches. LRU = Least Recently Used. Purges the oldest cache entries if there are more than the LRU List length entries every "rate" pages. This will keep space available in the cache for new seach results

LRU List Length

Applies to partial word searches. Limit the results cache to this number of entries. I'd say the 100 default here is pretty low. I set mine to 10000

Clear Search Words

Deletes all words from the partial word search cache

Delete Index

Removes the words in the search word index for the content type selected.

Delete and Rebuild Index

Deletes and rebuilds the index for the content type selected. This can take a really long time if you have a lot of content. There will be a command line tool available soon for those that need it. The page timeout will be set to 5 minutes during this operation - however, I have been told this may not be even close to the time needed if your site has thousands of pages.

After installing the new search module ...

After installing the new search module, you will have to do a reindex at least once on your site to get everything in there. (unless you have done this using CVS versions already). This hasn't been tested yet on a large data set - so beware, and backup.

Command Line Reindex

There is a ))cmd_line_reindex.php(( file under the search directory. It can be run from the command line in case you have a large site that needs a complete reindex. This way, web page time-outs don't interfere with the indexing.

It has been tested in Windows and Linux. If you get to a command prompt in the search directory, type:
php ))cmd_line_reindex.php((.

There are several options for cmd_line_reindex - all reasonable well documented in the source code.

Remaining Issues

Remaining issues that I'd like to see tackled:
  • Allow Stop Words
  • Allow "common" word list (to prevent indexing of common words)
  • Allow search to search more than liberty content - like phpbb

Page History
Date/CommentUserIPVersion
30 Jun 2008 (09:30 UTC)
Kozuch85.207.244.16012
Current • Source
Sean Lee71.254.8.12310
View • Compare • Difference • Source
Sean Lee71.254.3.1289
View • Compare • Difference • Source
Sean Lee71.161.201.1908
View • Compare • Difference • Source
Sean Lee71.161.201.1907
View • Compare • Difference • Source
Sean Lee71.161.201.1906
View • Compare • Difference • Source
Sean Lee71.161.201.1905
View • Compare • Difference • Source
Sean Lee71.161.201.1904
View • Compare • Difference • Source
Sean Lee71.161.201.1903
View • Compare • Difference • Source
Sean Lee71.161.201.1902
View • Compare • Difference • Source
SEWilco207.195.192.91
View • Compare • Difference • Source