It's been a busy week at Tourism! In addition to pushing out new web sites, my team has also been busy launching the next iteration of search on www.westernaustralia.com. Although wa.com is a MOSS-based web site, it uses a custom search system built in-house to meet an exacting set of requirements and keep half the planet happy.
Search 2, as it's known internally, started out as a retrofit of the tours search sub system but quickly evolved into a full rewrite to make search faster, more accurate, and more configurable while introducing new features like spelling suggestions.
Tourism WA has a paid membership model where tourism operators pay an annual fee to join the Tourism WA Network in return for a listing on westernaustralia.com. As a result, everyone wants to see their product listing on top of the pile when it comes to search results. We also have a lot of custom content on the web site: if you search for "Perth", the MOSS content page related to Perth is listed first; if you search for "hotel", you're probably after accommodation products but the display of the hotel listings in WA are randomised to ensure fairness. Trust me, these are some simple examples--what we end up with is a complex business logic that dictates how search results are weighted and ranked.
The search implementation on wa.com has a long and sordid history. We started out using the in-built MOSS search and the BDC, crawling MOSS page content and tourism product stored in the Australian Tourism Data Warehouse. The BDC is a great tool for unifying disparate data sources but MOSS search simply didn't give us the configurability required to deliver on the business requirements.
When panic set in, a Google search box was evaluated. While it did search, as you might expect, it only searched the Google way; it's still sitting in the server room, unused. As everyone fretted over the many problems we were facing, Rick, the developer working on search, went "underground" and built the RDC--or Rick's Data Catalog. The RDC would eventually replace the BDC and MOSS search in our little world. wa.com was launched with this solution, which is actually build on SQL Server Full Text indexing.
The RDC got us across the line after a never ending round of user acceptance testing but fell down in the tours category. It also abused some of the more horrific technologies from some of our older sites like XSLT and inline C#. It was also just a messy proof of concept at the code level and had some major design flaws. An example of the latter was a crawler that couldn't be scheduled and cleared the main index table before initiating the crawl; the problem with this approach was no search results would be returned while the crawler was run--apart from the manual administrator intervention required to actually kick it off and verify the results.
Search 2 is one of those engineering marvels you come across so rarely in the software world. The underlying design is fairly straightforward for what it does and it uses the chosen technology set appropriately--judiciously and as required. In other words, it's not over-engineered and gets on with the job in a logical, unobtrusive way. The grunty data work is relegated to the database where it belongs and the main UI is composed of configurable web parts.
That's not to say Search 2 didn't take a while to build. Nico, one of most experienced team members when it comes to MOSS and search, has been hard at work on this project since April. UAT progressed over a month and deployment took three days!
We're already building on top of the Search 2 fundamentals Nico has in place. An online booking engine has passed the proof of concept stages and a new itinerary planner should be out in the new year. Search 2 will also form the basis for a search re-release on the partner sites, which adds another few dimensions to the mix. Working with the Search 2 code base to integrate these components has almost been pleasurable. Either way, who can complain when all of the project stakeholders are happy?!
Nico's off for a well-deserved Christmas break and I'm quietly confident we won't have any major bedding in problems during the four weeks he's away (unless the database server dies but that's another story). Meanwhile, Search 2 should be humming happily along--have a look for yourself!
Custom-Built Microsoft Office SharePoint Server 2007 Branded Sites and Webpart Development - info@mediawole.com
Nice work mate, reading this blog post brought all the memories back. Feels like yesterday when I was working with the BDC & Search on the ATDW at Tourism WA, but it has been 2 and a half years!!!
ReplyDeleteYeah the business requirements were always going to be tough to implement ;o)
Tours especially, what I like to remember as the "freak" category of all ATDW product categories.
Kudos to Nico for finally implementing what sounds like the perfect solution, nice work fellas