Search Update

September 19th, 2009 | by bryan |

globeI haven’t had a chance to write much about it, but a couple weeks ago, I moved ReadPath’s search from a straight Lucene install with hand managed indices to Solr. I was a little hesitant at first because I though that since Solr was primarily an interface on top of Lucene that it would just slow things down and that I could manage things better. Boy was I wrong. I’m completely blown away by how easy it is to manage a search index with Solr. It handles caching and warming of indices. Date sorting is consistently faster and overall load on the search server is much lower.

Because things are running so much smoother, instead of doing a minor index build every other hour as I used to do, ReadPath is now updating the search every 2 mins. While not real-time per se it’s getting awfully close. After hooking up a real-time spidering system like pubsubhubbub or rsscloud things will be very near to actual real-time.


Post a Comment