Tokyo Performance

I'm on a month-long holiday back home at the moment (Australia -> Canada) so I shouldn't even be thinking about MOSS and work but nonetheless I am. This is day one into my time off and since I'm waiting in the North West Airlines lounge in Narita airport in Tokyo at the moment, I really don't have anything better to do (laptop left at home...). Specifically, I'm thinking performance.

In previous posts I've alluded to the performance issues we're addressing across the MOSS 2007 sites we maintain (www.westernaustralia.com, www.tourism.wa.gov.au, and www.australiasnorthwest.com with more to come before the end of the year). We've done a hell of a lot of work over the last few months to bring our page load times in line with internet "standards" (say, less than ten seconds) following continuing internal allegations the www.westernaustralia.com site was slow to load. 

The focus to date has been on reducing the number of requests it takes to load the home page (we started around 130 if you're viewing the site internationally and we're now down to 70 or 80), as well as trimming weight, and optimising our server environment. We use Gomez to measure relative page load times from various locations around the world and the results indicate latency causes huge problems for our sites. The wa.com site was designed to be "visually interactive" and by default as a tourism site it has to pop. Although a Sydney-based design firm did the graphic design, we had no requirements relating to performance apart from a useless mention to do with broadband users. Anyway, we've been testing the site through Gomez every hour since May and our global average home page load time started off somewhere around 44 seconds with spikes to 120 seconds in some regions. Ouch. By reducing requests, making sure a reasonable cache and compression policy was implemented, and doing the basics like reviewing image compression, we got load times down to the 36 mark before the asw.com site launched and started caning the shared database server. We've also signed up with Akamai (a content delivery network) and that should be live sometime next week.

So that's the background. And here I am in Tokyo loading the home page from a cold cache at around 2:30pm on a Saturday. Database issues aside, the page was useable and just about fully loaded by the 18-20 second mark. Japan has some amazingly fast connections and as Perth has a massive connection to Singapore I suppose it's possible my request was routed more or less directly across some fancy infrastructure. I know the site, of course, but my perception was it loaded in a reasonable timeframe. We're off from Tokyo in a few seconds to the US and then Canada so it will be very interesting to see whether the performance I just experienced is either Japan-specific or Saturday-specific (traditionally a busy time on our servers). 

Installing Visual Studio 2008

We finally got around to installing Visual Studio 2008 at work the other day and while everyone had a simple time of it on their clean new Vista machines (quad core, 4GB, 500GB, dual 22" monitors) my icky old XP SP2 laptop refused to play the game. Setup was repeatedly failing to install the Web Authoring Component. The .NET Framework 3.5 went in without hassle.

I tried uninstalling 3.5 and starting fresh, checked the event log and the install log but had nothing to guide me on my way. Error info was hard to come by but the Web Authoring Component installer was definitely faltering, identified by this: ISetupComponent::Pre/Post/Install() failed in ISetupManager::InternalInstallManager() with HRESULT -2147023293.

For the record, this machine has VS 2003, VS 2005, MCMS 2002, VSS 2005, SharePoint Designer 2007, and Office 2007 (including Visio and Project) installed.

I got it working in the end so here are a handful of tips to assist with your troubled install journey:

• Nothing of use gets logged to the event logs with this problem so don't bother looking there.
• The install log presented by the installer on failure is high level and won't tell you much.
• Additional install logs get dropped to your %temp% directory (in my case this was C:\Documents and Settings\mhanes\Local Settings\Temp). dd_error_vs_vstscore_90.txt identified the HRESULT error mentioned above but Googling this wasn't too helpful. dd_install_vs_vstscore_90.txt didn't hold much info. A MS site somewhere suggested clearing the contents of this directory before attempting to reinstall again but that didn't help in my case.
• SetupExe.log (from the same directory), however, was quite useful. It informed me, like others, that I had Office 2007 beta software installed on my machine (and this same problem most likely prevented me from installing Groove a few months back). The VS 2008 installer is apparently tied up with the Office 2007 installer or something like that. At the bottom of this file, I was told "Catalyst beta product conflict check failed. Office Beta product conflict is detected on the computer, productcode={30120000-00B2-0409-0000-0000000FF1CE}". As far I knew, I didn't actually have any Office beta software installed so I was puzzled by this one. The error message went on to say: "The 2007 Microsoft Office system does not support upgrading from a prerelease version of the 2007 Microsoft Office system. You must first uninstall any prerelease versions of the 2007 Microsoft Office system products and associated technologies." Fun.
• A few other bloggers out there seemed to have a similar problem and identified 30120000-00B2-0409-0000-0000000FF1CE as the Compatibility pack for the 2007 Office System. M'kay, so I uninstalled it the normal way (early versions were based on beta software and I'd had it installed for ages so this was a likely trouble-maker). Reinstall, no luck, same error message. Running this: MsiExec.exe /X{30120000-00B2-0409-0000-0000000FF1CE } also indicated this thing wasn't installed.
• A few bits of info also suggested investigating HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall so I peeked in there at the 30120000-00B2-0409-0000-0000000FF1CE key and it was identified as Office 2007 Add-in - Microsoft Save as PDF or XPS (Beta). Eureka! Sort of. I uninstalled this in the normal way, re-ran setup, but still no luck--same error message.
• I finally tried MsiExec.exe /x {30120000-00B2-0409-0000-0000000FF1CE} and this time it had some positive feedback.

And the VS 2008 install finally worked! It was even quick about it!

Authored Links Don't Update Between Environments

We have yet to install the latest and greatest content deployment update (the so-called Infrastructure Update) but I think we've pretty much given up on content deployment ever working with the custom controls and web parts used on www.westernaustralia.com. To this end we built our new Tourism Western Australia "partners" solution on the concept of a single environment used for both editing and anonymous access--no more disconnected authoring and production.

So far so good with this solution: we've got the first of many sites out the door (www.australiassouthwest.com) and there's been no looking back despite some unrelated issues. That said, we have come across one very odd problem with authored links not updating from http://edit to https://edit to http://www. We use a reverse proxy in production and do some content rewriting to ensure any stray "edit" anchor tags are rewritten as "www" but as our remote content editors access the secure site, any content links and images created internally on http://edit forever refer back to that site. Relative links don't help in this case because SharePoint helpfully changes relative URLs to their absolute equivalent.

I've isolated the problem to the custom columns or a custom content type deriving indirectly from ArticlePage. We provision these columns and content types via features. Existing columns inherited from the base content type are updated automatically between environments by SharePoint. To top it all off, adding a new column to the existing content type works fine and links are updated correctly in that column.

I've reviewed our feature code and can't even think of a mechanism for disabling this functionality. We're using similar techniques on the westernaustralia.com site (which until recently was RTM plus a bunch of hotfixes while the partners environment was built as SP1) and have no issues in this area when moving content between out authoring and production environments.

I've come across a single technet forum post with a similar problem and the MSFT respondent suggested a case be logged with MS... I imagine we'll soon be taking that path ourselves.

[Update: We raised a case and MS got to the bottom of the problem on our behalf. Here's the solution.]

Uncustomising MOSS Pages - Part 2

The other day I wrote about the frustrations of uncustomising a page layout deployed via a feature to the master page gallery. I've since done some additonal hunting and come across the SPFile.RevertContentStream ( ) method to revert a page layout or master page to its original state so feature deployments work again.

Calling this successfully reverts/uncustomises/reghosts the file in question, although I did notice the version history accessible through the master page gallery remains as-is. Nevertheless, a change to the file system .aspx file after running this code is honoured by MOSS.

Here's the code I wrote to encapsulate that call (it's a bit rough but for a one-off excercise does the trick). I simply run this from a console application (which you can download from here).

// Wait for debugger to attach
Console.WriteLine ("Waiting for debugger to attach. Press any key to continue...");
Console.ReadLine ();

const string FILE_PATH = "/_catalogs/masterpage/ThreeColumns.aspx";

using (SPSite site = new SPSite ("http://dev:30000/"))
{
    using (SPWeb web = site.AllWebs ["/"])
    {
        SPFile file = web.GetFile (FILE_PATH);

        Console.WriteLine ("File status before change: " +
                            file.CustomizedPageStatus);

        if (file != null &&
            file.Exists &&
            file.CustomizedPageStatus == SPCustomizedPageStatus.Customized)
        {
            Console.WriteLine ("Reverting file...");

            try
            {
                file.RevertContentStream ();
            }
            catch (Exception e)
            {
                Console.WriteLine (e);
            }

            Console.WriteLine ("File status after change: " +
                                file.CustomizedPageStatus);
        }
        else
        {
            Console.WriteLine ("Unable to retrieve file");
        }
    }
}

Uncustomising MOSS Pages

The newly-launched www.australiassouthwest.com site is live but suffers a small problem: at some point during development, after we imported content from MCMS 2002, someone or something checked out and published our trusty three column page layout. We deploy our master pages and page layouts using solutions and features, which means those artefacts live in the server's file system and are referenced by MOSS directly. Checking out and publishing one of these pages "customises" or "un-ghosts" the page by effectively moving a copy to the content database; changes made to the original file system content (ie. during a subsequent deployment) are then ignored.

As far as I've been able to determine, it's impossible to roll back to the original version 1.0, although you can sort of do it by checking out the page again, restoring v1.0, and publishing it as v3.0. This will pick up the latest file system content but any future changes won't be automatically reflected. Today I found a system page which does exactly the same thing: /_layouts/reghost.aspx. As the name implies, the end result for the page, page layout, or master page specified is a re-ghosted, or uncustomised, version of the page. I ran this with the page checked out and logged in to the site as administrator so my version update was logged as System Account (it failed using my own domain credentials).

All well and good but I'm still not back to a clean starting point and I think the next easiest option is to create a copy of the page layout in question and point all content pages to that layout. But that's really ugly.

[Update: Here's my solution]

Loving Google Analytics

I'm absolutely loving Google Analytics. The interface is incredibly responsive and the wealth of information made freely available is truly awesome. To date GA has yet to prevent me accessing a particular bit of data the way I want it whereas Omniture's ability to drill into specific reports is stymieing at times. The menu is simply intuitive where I often find it difficult to find specific Omniture reports in their convoluted menu structure.

It's so interesting to know where you're coming from, how you're arriving here, and which posts attract the most attention! Thanks for reading!

Google Analytics vs Omniture

We use Omniture's Site Catalyst tools for web site analytics across www.westernaustralia.com and all of the connected Tourism Western Australia eMarketplace sites. There's been a mad rush on stats lately, initiated by the CEO, so I've been waist deep in Omniture stats coming to terms with an area of web development to which I've never paid much attention.

Omniture costs money and until the other day I was under the impression Google Analytics was a paid service as well. Silly me. Until now I've been displaying AdSense ads on this blog to get a rough idea of page views and not a lot else. You may have noticed I don't care much for AdSense.

I finally decided to have a look at Analytics and after realising it is a free service I signed up. Much to my suprise, the stats available are very comprehensive and while I imagine Analytics is taregetting a different group of sites than Omniture, I'm struggling to see many advantages to going with an Omniture solution. Customisation would be a biggie and I've heard Analytics stats are big shonky at times, which is kind of important ;-)

Analytics is now live on this blog via a simple bit of JS and I'm dying to know how you're arriving here and how often you return. It's all fascinating stuff and I've been watching my "readership" increase slowly but surely over the last year so some additional detail will be welcome.

MS WCM Resource Centers Now Online

Microsoft have been kind enough to pull together a bunch of MOSS WCM stuff in two central locations:

• http://technet.microsoft.com/en-us/office/sharepointserver/bb736743.aspx (for operations)
• http://msdn.microsoft.com/en-us/office/bb251752.aspx (for devs)

Value does not fall within the expected range.

This is another very annoying, useless MOSS error message. See below for a possible solution.

Server Error in '/' Application.

Value does not fall within the expected range.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.ArgumentException: Value does not fall within the expected range.

[ArgumentException: Value does not fall within the expected range.] Microsoft.SharePoint.Library.SPRequestInternalClass.GetFileAndMetaInfo(String bstrUrl, Byte bPageView, Byte bPageMode, Byte bGetBuildDependencySet, String bstrCurrentFolderUrl, Boolean& pbCanCustomizePages, Boolean& pbCanPersonalizeWebParts, Boolean& pbCanAddDeleteWebParts, Boolean& pbGhostedDocument, Boolean& pbDefaultToPersonal, String& pbstrSiteRoot, Guid& pgSiteId, UInt32& pdwVersion, String& pbstrTimeLastModified, String& pbstrContent, Byte& pVerGhostedSetupPath, UInt32& pdwPartCount, Object& pvarMetaData, Object& pvarMultipleMeetingDoclibRootFolders, String& pbstrRedirectUrl, Boolean& pbObjectIsList, Guid& pgListId, UInt32& pdwItemId, Int64& pllListFlags, Boolean& pbAccessDenied, Guid& pgDocId, Byte& piLevel, UInt64& ppermMask, Object& pvarBuildDependencySet, UInt32& pdwNumBuildDependencies, Object& pvarBuildDependencies, String& pbstrFolderUrl, String& pbstrContentTypeOrder) +0 Microsoft.SharePoint.Library.SPRequest.GetFileAndMetaInfo(String bstrUrl, Byte bPageView, Byte bPageMode, Byte bGetBuildDependencySet, String bstrCurrentFolderUrl, Boolean& pbCanCustomizePages, Boolean& pbCanPersonalizeWebParts, Boolean& pbCanAddDeleteWebParts, Boolean& pbGhostedDocument, Boolean& pbDefaultToPersonal, String& pbstrSiteRoot, Guid& pgSiteId, UInt32& pdwVersion, String& pbstrTimeLastModified, String& pbstrContent, Byte& pVerGhostedSetupPath, UInt32& pdwPartCount, Object& pvarMetaData, Object& pvarMultipleMeetingDoclibRootFolders, String& pbstrRedirectUrl, Boolean& pbObjectIsList, Guid& pgListId, UInt32& pdwItemId, Int64& pllListFlags, Boolean& pbAccessDenied, Guid& pgDocId, Byte& piLevel, UInt64& ppermMask, Object& pvarBuildDependencySet, UInt32& pdwNumBuildDependencies, Object& pvarBuildDependencies, String& pbstrFolderUrl, String& pbstrContentTypeOrder) +215 Microsoft.SharePoint.SPWeb.GetWebPartPageContent(Uri pageUrl, PageView requestedView, HttpContext context, Boolean forRender, Boolean includeHidden, Boolean mainFileRequest, Boolean fetchDependencyInformation, Boolean& ghostedPage, Byte& verGhostedPage, String& siteRoot, Guid& siteId, Int64& bytes, Guid& docId, UInt32& docVersion, String& timeLastModified, Byte& level, Object& buildDependencySetData, UInt32& dependencyCount, Object& buildDependencies, SPWebPartCollectionInitialState& initialState, Object& oMultipleMeetingDoclibRootFolders, String& redirectUrl, Boolean& ObjectIsList, Guid& listId) +1553 Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.FetchWebPartPageInformationForInit(HttpContext context, SPWeb spweb, Boolean mainFileRequest, String path, Boolean impersonate, Boolean& fGhostedPage, Byte& verGhostedPage, Guid& docId, UInt32& docVersion, String& timeLastModified, SPFileLevel& spLevel, String& masterPageUrl, String& customMasterPageUrl, String& webUrl, String& siteUrl, Guid& siteId, Object& buildDependencySetData, SPWebPartCollectionInitialState& initialState, String& siteRoot, String& redirectUrl, Object& oMultipleMeetingDoclibRootFolders, Boolean& objectIsList, Guid& listId, Int64& bytes) +692 Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.FetchWebPartPageInformation(HttpContext context, String path, Boolean impersonate, Boolean& fGhostedPage, Byte& verGhostedPage, Guid& docId, UInt32& docVersion, String& timeLastModified, SPFileLevel& level, String& masterpageUrl, String& customMasterPageUrl, String& webUrl, String& siteUrl, Guid& siteId, Object& buildDependencySetData) +132 Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.GetWebPartPageData(HttpContext context, String path, Boolean throwIfFileNotFound) +773 Microsoft.SharePoint.ApplicationRuntime.SPVirtualFile.GetFile(String virtualPath, Boolean fetchContent) +78 Microsoft.SharePoint.ApplicationRuntime.SPVirtualFile.GetFile(String virtualPath) +30 Microsoft.SharePoint.ApplicationRuntime.SPVirtualPathProvider.GetFile(String virtualPath) +172 System.Web.Hosting.VirtualPathProvider.GetFile(String virtualPath) +18 System.Web.Hosting.VirtualPathProvider.GetFileWithCheck(String virtualPath) +11 System.Web.FormatterWithFileInfo.GetSourceFileLines(String fileName, Encoding encoding, String sourceCode, Int32 lineNumber) +229 System.Web.DynamicCompileErrorFormatter.get_MiscSectionContent() +926 System.Web.ErrorFormatter.GetHtmlErrorMessage(Boolean dontShowSensitiveInfo) +818 System.Web.HttpResponse.WriteErrorMessage(Exception e, Boolean dontShowSensitiveErrors) +645 System.Web.HttpResponse.ReportRuntimeError(Exception e, Boolean canThrow, Boolean localExecute) +271 System.Web.HttpRuntime.FinishRequest(HttpWorkerRequest wr, HttpContext context, Exception e) +338

While the browser output is next to useless and almost misleading, the Application event log may contain additional information of use (i.e. the real source of the exception).

Dirty Words (Michael Hanes) - blog.mediawhole.com mailto:info@mediawole.com 

Direct Dependencies Limit with User Controls

The westernaustralia.com web site uses a large number of user controls hard coded across twenty-odd page layouts--a big no-no. While we've matured as a dev team since this site launched and now fully appreciate the value of web parts and a small set of key page layouts, unfortunately the site is live as-is.

From a production support perspective we're still working on the site and adding new user controls here and there. This shouldn't be problematic but MOSS, in it's infinite wisdom, limits the number of user controls on a page and fails when that limit is reached.

Server Error in '/' Application.
Parser Error

Description: An error occurred during the parsing of a resource required to service this request. Please review the following specific parse error details and modify your source file appropriately.

Parser Error Message: The page '/_catalogs/masterpage/Home.master' allows a limit of 11 direct dependencies, and that limit has been exceeded.

Luckily, this is easy to work around via config. Each MOSS web application uses its own web.config file (a special, MOSS-extended version). In it you'll find the safeMode element which has a DirectFileDependencies attribute. This is set to 10 by default so adding more than ten user controls to a page layout will trigger the above exception. Simply increase the value to whatever you need and away you go. In my dev environment I've set it as high as 25 or 30 with no apparent issues although in production we set it where it needs to be (just over).