Perth SharePoint User Group Webcast Q&A

Jeremy sent through a handful of questions from the February PSPUG webcast so I thought I would answer them here. Thanks to Joshua for these awesome questions!

Q. What decisions did you make early that you wish you could change now? Or what would you do differently if you could start over?

A. Where to begin?! We’ve been lucky, in one sense, having the option to rebuild the partner web sites on a separate platform. At that time, westernaustralia.com was up and running and we pretty much did start over.

On the wa.com side, we made a very conscious decision soon after launch to move from SharePoint Designer and master pages/layouts held in the content database to a solution/feature-based deployment situation. Moving away from SPD is one of the best things we’ve ever done. We’ve currently got one feature for deploying most things on the site but there are cases where it would have been useful to componentise some aspects.

The biggest issues we face currently, off the top of my head, are as follows:

• Too many page layouts I mentioned the concept of page layout sprawl in the presentation; we’ve locked ourselves into a situation where content editors love being able to easily create a new page with all the page components they need for that section of the site at the cost of flexibility from their end and maintainability from the dev perspective. We’re revisiting this now and looking at ways a content editor can create a new page from a smaller selection of layouts and have the necessary web parts, specific to that subsite, added automatically. The Andrew Connell way of doing this through the AllUsersWebPart node in manifest.xml doesn’t work very well in this case as it’s too coarse and results in duplicate web parts every time the related features are activated. I imagine we’ll probably end up using the SPLimitedWebPartManager to do this but I’m not yet sure how!
• Reliance on user controls over web parts This very much ties in with my first bullet point. User controls are easier to develop because you’ve got real markup to play with and they help construct a tight, enforceable template but you’re trading configurability and the web part storage mechanism to go down that path. The partner sites are based solely on web parts and they work quite well in that environment where different partner sites want web parts in different places. Writing web parts that are centrally configured is also an interesting twist to the web part approach.
• Separate authoring farm I’ll be speaking more about this in part two of my presentation but there was a perceived need (on recommendation from MS and for records keeping regulations) to create a separate farm for content editing activities; content would then flow happily from authoring to production using the content deployment tools. Yeah right. In reality we have no need for a separate farm and with a lot of new hardware going in shortly we’ll be looking to merge the authoring and production farms. We’re doing this successfully with the partner sites.
• Two code bases We maintain two separate Visual Studio solutions, one for wa.com and one for partners. The partner site development effort taught us how to really use the SharePoint platform as a site provisioning platform and while wa.com and the partner sites are quite different, I’d like to see the two solutions merged so wa.com becomes just another partner site, perhaps with it’s own special set of master pages/layouts.
• Lists Some things started out as lists on wa.com but we drifted away from lists for a while for performance and cacheability reasons. We’ve worked through those issues now and use lists successfully on the partner sites to simplify per-site, content editor-managed configuration and could easily apply that knowledge back to wa.com.
• Abstraction The SharePoint learning curve coupled with my predecessor’s minimalist approach to all things code mean our code base doesn’t abstract access to the platform. As a result, it’s currently up to every developer to understand SharePoint integration and things like how to correctly dispose of unmanaged objects. Good stuff to know, no doubt, but it makes life more difficult for a new developer coming on board who hasn’t worked with MOSS and means different devs end up doing things differently, which complicates maintenance.
• Variations The variations piece seemed appropriate at the time and the wa.com site structure reflects our variation structure. Variations actually work but they work at too high a level four our specific needs. Some of our pages are not to be overwritten while others should be and unless you’re using workflow to stop it happening, the variations mechanism will simply copy the entire site (AFAIK—it’s been a long time since we last looked at this one!). In reality, while our marketing division likes their language-based site structure, they’re not willing to maintain different content for each site so it’s complete overkill.

Q. Could you explain in more detail the search tool you built? Seems to have a nice combination of BDC [Business Data Catalog] and SQL features.

A. I was hoping to have our “search dude” along to speak on this subject but unfortunately he was too busy learning to fly—to give you an idea of his calibre. I’ll tell you what I know.

I may have briefly mentioned in the presentation that we started out, on Dimension Data’s recommendation, using MOSS search and the BDC. It’s worth pointing out we also used the SharePoint v2 search bits in conjunction with MCMS 2002 on the old sites so it seemed like a logical step forward. For the record, we’re indexing MOSS page content, tourism product in the ATDW database, and .pdfs.

With the weight of many different tourism operators and industry bodies seeking to influence their ranking in our search results, our search requirements are convoluted. As a rough example, we always display pages first, followed by accredited tourism operators (randomised), followed by non-accredited operators (randomised), followed, until recently, by non-members (randomised). In many cases, this resulted in the need to boost certain keywords, boost individual results in special circumstances, and “band” results. Then there’s things like B&B versus Bed and Breakfast, returning accurate results for “perth hotels five star”, and so on.

The guys working with the Beta 2 TR found it didn’t provide the customisation we needed and had some major memory management issues; despite assistance from the SharePoint Ranger Team, we were soon looking to alternatives. The Google search appliance was examined but we found even Google’s search algorithm wasn’t suited to our particular business domain. We haven’t revisited the BDC or Google since so these comments should not be considered a reflection on the current products.

With the BDC bug out of our bonnet, it was time to look at a custom build and that was accomplished by following the BDC model and adjusting it to our needs. The solution we’ve got in place now is a three-part system comprising a crawler, a SQL Server 2005 database which serves as an index, and a presentation layer built with web parts. The crawler is a console application that uses the MOSS API to iterate over the SharePoint sites it’s configured to crawl; it also has separate functionality to crawl the ATDW database. The crawler can do full or incremental crawls and the latest build has been working flawlessly.

The crawler doesn’t duplicate page and product record data but collects specific, searchable data that is then stored in the database. The database structure was originally de-normalised but performance testing during the latest rebuild found performance was better when the data was restructured according to typical relational database design principles. The database tables are also full-text indexed and a number of stored procedures expose results to the application using the business logic discussed previously.

While this solution meets our specific needs, it must be stressed the database structures reflect the things we index: pages, tourism products, and files. In other words, this solution works really well for us but it’s not a rebuild of the BDC and pointing it to a different LOB database would require a significant amount of rework.

Q. Could you profile a typical content author / publisher? What training do they do?

A. We’ve got two different types of “typical” content editor.

On the westernaustralia.com side, content editors have full control over the site (the security model is admittedly weak) and have a marketing/digital marketing background. These guys are the business owners of the site and they’ve been working with the platform and the site for a couple of years. They’re pretty comfortable creating new pages from an overly large selection of layouts, checking out an existing page, adding reusable content and rich text, checking in/approving/publishing. Basically, they’re young, web-savvy types who call themselves geeks because they know how to cut basic HTML and use Flickr ;-)

We used to have a dedicated trainer in house (no more) and while some related training materials were prepared and delivered at the time, I think the marketing team now learn it as they go and share information between themselves.

On the partners side, the experience of these content editors varies widely. Some may have worked with MCMS previously so they’ve got some preconceived ideas about how content management systems work (and how responsive they should be!). These guys all work off-site but are supported via a 1800 number and our four-person Service Delivery and Content Management team—effectively a content helpdesk. The partners range from youngish types focused solely on maintaining content to older volunteers; they all have different levels of experience using the internet. SDCM therefore train all new partners, bringing them in and going through everything they need to know to maintain the content on their sites. This includes page basics as well as managing list content, accessing stats, and so on.

Hope this was useful… any additional questions posted as comments shall be addressed here.