How to pass JSON arrays and other data types to an ASMX web service

Ah interoperability… great fun, great fun.

So jQuery is your new best friend and, along with JSON, there's nuthin' you can't do. The server side stuff is still there in the background and you've got some old school ASP.NET (.asmx) web services hanging around but DOM elements are otherwise flying all over the place, postbacks are just so passé, and even the marketing girls are mildly impressed at your skillz. You're branching out, shifting code and complexity from the server to the browser, and it's time to do some heavier data shunting. Here a few things to know about passing JSON data to an ASMX web service that may help you on your way…

JSON.stringify

Know it, use it, love it. It's part of the JSON2 library and you need it if you don't have it already. Use it to prepare (aka properly encode) your JSON data before sending it off to the big mean ol' web server:

data: {"days": JSON.stringify(["Mon", "Tues"])}

That will encode as &days=["Monday","Tuesday"]

Yeah, I know, it's another file to download but the guy who wrote JSON also wrote this and it can be merged and minified. I've tried writing my own mini-version as a function and while this works for simple strings, save yourself some time when it comes to arrays and the like and just use this sucker.

Arrays

Arrays seem trickier than they should… maybe I'm just a dumb guy—probably. Anyway, you can pass a JSON array to an .asmx web service without much work at all.

The client-side call listed above is everything you need to do from that end. On the server side, create yourself a new web service method with a List<string> parameter:

[WebMethod]
[ScriptMethod (UseHttpGet = true, ResponseFormat = ResponseFormat.Json)]
public string ConsumeArray (List<string> days)
{…}

That's all there is to it. If you're not passing in strings, declare the List<> parameter with a type of object or something else. You can use .NET arrays in the web method signature as well if you really want (need) to.

Integers

When in doubt, stringify:

data: { "i": JSON.stringify(2) }

An int parameter on the web service end will handle this graciously.

Booleans

The good ol' boolean—a simple concept computer science has managed to bastardise like no other…

When in doubt, stringify:

data: { "b": jsonpEncode("true") }

Like the int parameter, a bool in your web method signature will take care of this.

A brief note: JSON, or rather jQuery's parseJSON function, is a particular beast and doesn't seem to know about anything other than the lower case true and false strings. If, for any reason, you ToString a bool in your .NET web service and try to pass it back, parseJSON will fail. If you forget to brush your teeth in the morning, parseJSON will fail.

Dates

Sorry, on my todo list ;-)

Tools

When working through this stuff, it pays to have Fiddler open to inspect the requests you're sending through and any error messages you're getting back. I find Fiddler sometimes breaks this stuff so try turning off the capture if you're getting weird errors; optionally, revert to Firebug (Firefox only, of course).

Fully decoding the data you sniff from a JSONP request passed along in the query string will require some additional tooling; in short, you'll want to decode the value using a free online tool like Opinionated Geek's URL decoder.

If you found this post helpful, please support my advertisers.

Exposing the Global Assembly Cache

If you're familiar with the Global Assembly Cache (GAC) you're probably aware there's a special file system viewer thingy (technical term) sitting over top of the GAC contents at c:\windows\assembly; this is a nice convenience when it comes to registering assemblies in the GAC—simply drag and drop, avoiding a trip to the command line and gacutil –i

More often than not, this is all good. When you need to dive into the real GAC, to extract an assembly, drop in debugging symbols, or whatever, you'll quickly realise the viewer is somewhat limiting.

To get past the GAC's outer facade, you've got a few options:

• From a command line, browse to c:\windows\assembly\gac_msil
• Map a network drive to \\machine-name\c$\windows\assembly\gac_msil
• Create a virtual drive: subst z: c:\windows\assembly\gac_msil where 'z:' is any unmapped drive letter
• Start –> Run c:\windows\assembly\gac_msil
• Turn off the viewer altogether to browse the GAC directory structure normally within Windows Explorer: create a new DWORD named DisableCacheViewer with a value of 1 below the HKLM\Software\Microsoft\Fusion key

Five ways to do the same thing? Well this is Windows after all—and there are probably more!!! ;)

If you drop the gac_msil bit you'll find there are other directories that make up the GAC proper to explorer but most of what you'll be after resides below gac_msil. Each assembly is represented in by name as a folder with different versions represented as sub folders named as the version number with a GUID appended; the assembly proper will live in one of these folders.

If you found this post helpful, please support my advertisers.

Changing a VS2010 SharePoint package name

The SharePoint development tooling built into Visual Studio 2010 does a pretty good job at hiding some of the ugly bits involved in creating a SharePoint solution file. No longer must we deal with ddf files (hooray!) but we also lose some control (or, at the very least, have to dig a bit further to change some things that were previously "easy" in the broader context of "really painful").

One of those things that we previously had full control over was the name of the solution file (.wsp) that results from our ddf file and the mighty makecab.exe. While the file name is now set from within Visual Studio, you have to open the package designer before being presented with the package properties:

Simply clicking on the Package.package file will present you with the file properties which is not what you want—you have to double-click/open the package designer.

The one thing I used to do with the ol' ddf file was specify the extension as .cab instead of .wsp. This was a convenience thing—SharePoint doesn't care about the extension (or at least doesn't object to .cab files)—but a .cab file will open in Windows Explorer on machines where you don't have a better archive tool installed like my favourite, 7zip. Essentially, it just saved me having to rename the file by hand.

Unfortunately I have yet to find a way to do this with VS2010; the package name is only that—Visual Studio appends ".wsp" to the end of whatever name you supply and I can't find a way to override that behaviour. In truth I'll probably just start using .wsp like everyone else but it may be possible to tweak the file name in one of the pre/post events.

If you found this post helpful, please support my advertisers.

.NET 4.0 application compatibility

Looks like those of us running SharePoint may be stuck on .NET 3.5 SP1 for a while longer but that's no excuse not to have .NET 4.0 installed side-by-side for use by other apps. One of the guys on the team, for instance, recently updated the custom crawler we run and needed pull in some of the Entity Framework features only made available in the 4.0 release; of course Microsoft will tell you the various .NET runtimes can be installed to run side-by-side and we managed to get away with a lot during the whole 2.0/3.0/3.5 onion thing.

Of course this all gets interesting very quickly when you've got a console application built against 4.0 that references assemblies (i.e. the SharePoint assemblies) built against version 2.0. Naturally everything worked fine in dev but as soon as we hit UAT, this exception cropped up:

An error has occured: Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information. [sic]

To address this, a simple .config element was added:

<configuration>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" />
  </startup>
…

Notably, the useLegacyV2RuntimeActivationPolicy="true" attribute was required and we found this out the hard way by not having in place initially (dodgy internet code most likely!).

If you found this post helpful, please support my advertisers.

Detection of product '{90140000-104C-0000-1000-0000000FF1CE}', feature 'PeopleILM', component '{1C12B6E6-898C-4D58-9774-AAAFBDFE273C}' failed

After sorting out a problem with the FIM Service not starting automatically after a reboot and several resultant application event log errors, there was one more thing to clean up: running a user profile sync would spit a couple of MsiInstaller warnings about product detection failing (see below).

Various users in the forums suggested granting the Network Service account read access to the C:\Program Files\Microsoft Office Servers\14.0\Service directory and/or the C:\Program Files\Microsoft Office Servers\14.0\Sql directory; I initially found only the latter was required (this directory currently has Read & execute, List folder contents, and Read) but on reboot, the same warnings were logged again. Granting the same access to \Services had the same result and I've found starting a crawl produces these warnings but they go away with subsequent crawls. Reboot and they're back :-(

This change relates to the following warnings logged as a crawl is initialised:

Log Name:      Application
Source:        MsiInstaller
Date:          2/09/2010 3:40:24 PM
Event ID:      1004
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
Detection of product '{90140000-104C-0000-1000-0000000FF1CE}', feature 'PeopleILM', component '{1C12B6E6-898C-4D58-9774-AAAFBDFE273C}' failed.  The resource 'C:\Program Files\Microsoft Office Servers\14.0\Service\Microsoft.ResourceManagement.Service.exe' does not exist.

Log Name:      Application
Source:        MsiInstaller
Date:          2/09/2010 3:40:24 PM
Event ID:      1001
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
Detection of product '{90140000-104C-0000-1000-0000000FF1CE}', feature 'PeopleILM' failed during request for component '{9AE4D8E0-D3F6-47A8-8FAE-38496FE32FF5}'

Log Name:      Application
Source:        MsiInstaller
Date:          2/09/2010 3:40:24 PM
Event ID:      1015
Task Category: None
Level:         Warning
Keywords:      Classic
User:          NETWORK SERVICE
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
Failed to connect to server. Error: 0x80070005

If you found this post helpful, please support my advertisers.

Forefront Identity Manager Service fails to start after reboot

Update [28/09/2010]: Spence recently released a follow-up article to the Rational Guide… in which he discusses an additional change for those of us using SQL Server aliases. Check out the section entitled "Using a SQL Server Named Instance" and scoot down to the local DTC configuration steps. I haven't tried this yet myself but it sounds promising.

Update [27/10/2010]: I see Spence has updated the above-mentioned article to include a section about this problem which validates the solution presented here.

After following Spence Harbar's Rational Guide to implementing SharePoint Server 2010 User Profile Synchronization, I was able to not only get the UPS service started but I was also able to run a sync on my first attempt. I probably got lucky ;-)

The one small hiccup I had along the way was getting the Forefront Identity Manager Service to start following a reboot; the service simply refused to start automatically despite being configured by SharePoint/FIM to do so. Interestingly, both the User Profile Service and the User Profile Synchronization Service items listed in Central Admin's Services on Server page listed the services as running. Starting the FIM Service manually from the Windows Services snapin succeeded (I didn't try directly through CA) but felt hacky and annoying.

What to do? Since the Synchronization Service was starting successfully and I could manually start the service after logging in, I assume this has to be some kind of dependency issue between the services themselves or SQL Server (some of the event log error message listed below definitely take issue with SQL).

Update 29/09/2010: After examining the sequence of event log entries relating to MSSQLSERVER and FIM, I can clearly see SQL is NOT ready to accept client connections by the time the FIM services kick in. I should point out my test environment is running as a single-server farm (AD, SQL, IIS, SharePoint, etc) so I'd definitely pay attention to Spence's follow-up article I note above in the 28/09 update.

My solution was to therefore set both services to start automatically at boot time after a delay by reconfiguring the startup type of BOTH services and Automatic (Delayed Start) in the Windows Services snapin:

Interestingly, I found the FIM Service starts before the FIM Sync Service, fwiw. I also still have one error remaining stating The Forefront Identity Manager Service cannot connect to the SQL Database Server but it doesn't prevent the services from starting or a sync from running.

So is this an inappropriate change to make? I can't say, especially with everyone and their dog saying "let SharePoint manage these services, don't start 'em manually!" In a single-server environment, I'll suggest it is acceptable. I know for certain both services now start automatically after a minute or so (once all other services set to just Automatic have started) and I can still run a profile sync; the following errors are also no longer present:

Log Name:      Application
Source:        Forefront Identity Manager
Date:          3/09/2010 12:37:17 PM
Event ID:      3
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: Cannot open database "Sync DB" requested by the login. The login failed.
Login failed for user 'DEV\SVC_SPFARM'.
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)
   at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject)
   at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart)
   at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject)
   at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
   at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
   at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
   at System.Data.SqlClient.SqlConnection.Open()
   at Microsoft.ResourceManagement.Data.DatabaseConnection.Open(SqlConnection connection)

Log Name:      Application
Source:        Forefront Identity Manager
Date:          3/09/2010 12:37:17 PM
Event ID:      3
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: Cannot open database "Sync DB" requested by the login. The login failed.
Login failed for user 'DEV\SVC_SPFARM'.
   at Microsoft.ResourceManagement.Data.Exception.DataAccessExceptionManager.ThrowException(SqlException innerException)
   at Microsoft.ResourceManagement.Data.DatabaseConnection.Open(SqlConnection connection)
   at Microsoft.ResourceManagement.Data.DatabaseConnection.Open(DataStore store)

Log Name:      Application
Source:        Microsoft.ResourceManagement.ServiceHealthSource
Date:          3/09/2010 12:37:17 PM
Event ID:      26
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
The Forefront Identity Manager Service was not able to initialize a timer necessary for supporting the execution of workflows.

Upon startup, the Forefront Identity Manager Service must initialize and set a timer to support workflow execution.  If this timer fails to get created, workflows will not run successfully and there is no recovery other than to stop and start the Forefront Identity Manager Service.

Restart the Forefront Identity Manager Service.

Log Name:      Application
Source:        Microsoft.ResourceManagement.ServiceHealthSource
Date:          3/09/2010 12:37:17 PM
Event ID:      2
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
The Forefront Identity Manager Service could not bind to its endpoints.  This failure prevents clients from communicating with the Web services.

A most likely cause for the failure is another service, possibly another instance of Forefront Identity Manager Service, has already bound to the endpoint.  Another, less likely cause, is that the account under which the service runs does not have permission to bind to endpoints.

Ensure that no other processes have bound to that endpoint and that the service account has permission to bind endpoints.  Further, check the application configuration file to ensure the Forefront Identity Manager Service is binding to the correct endpoints.

Log Name:      Application
Source:        Forefront Identity Manager
Date:          3/09/2010 12:37:17 PM
Event ID:      3
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: Cannot open database "Sync DB" requested by the login. The login failed.
Login failed for user 'DEV\SVC_SPFARM'.
   at Microsoft.ResourceManagement.Data.Exception.DataAccessExceptionManager.ThrowException(SqlException innerException)
   at Microsoft.ResourceManagement.Data.DatabaseConnection.Open(SqlConnection connection)
   at Microsoft.ResourceManagement.Data.DatabaseConnection.Open(DataStore store)
   at Microsoft.ResourceManagement.Data.TransactionAndConnectionScope..ctor(Boolean createTransaction, IsolationLevel isolationLevel, DataStore dataStore)
   at Microsoft.ResourceManagement.Data.TransactionAndConnectionScope..ctor(Boolean createTransaction)
   at Microsoft.ResourceManagement.Data.DataAccess.RegisterService(String hostName)
   at Microsoft.ResourceManagement.Workflow.Hosting.HostActivator.RegisterService(String hostName)
   at Microsoft.ResourceManagement.Workflow.Hosting.HostActivator.Initialize()
   at Microsoft.ResourceManagement.WebServices.ResourceManagementServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses)
   at Microsoft.ResourceManagement.WindowsHostService.OnStart(String[] args)

Log Name:      Application
Source:        Microsoft Resource Management Service
Date:          3/09/2010 12:37:17 PM
Event ID:      0
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      dev-sps2010-01.dev.mediawhole.com
Description:
Service cannot be started. System.Data.SqlClient.SqlException: Cannot open database "Sync DB" requested by the login. The login failed.
Login failed for user 'DEV\SVC_SPFARM'.
   at Microsoft.ResourceManagement.WindowsHostService.OnStart(String[] args)
   at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)

If you found this post helpful, please support my advertisers.

Troubleshooting user profile sync via FIM

Your initial foray into SharePoint 2010 user profile sync will likely lead you to the FIM client and, if you're anything like me, your mind will boggle at what FIM is, why it has to be involved at all, and where to start when things to horribly wrong.

I won't attempt to enlighten you on the first two subjects but I do want to point out some interesting and non-intuitive FIM user interface screens you may not be aware of and that will help you determine if your UPS setup is on the right path. By the way, you can run the FIM client as soon as the two FIM services are running on your machine (in other words, as soon as UPS has been provisioned but before you run a sync).

If you've got the UPS service in a running state, the next thing you'll likely want to do is run your first (or 50th) sync; in addition to the dodgy status screen within Central Admin itself, you can fire up the FIM client to watch from the bushes (the Operations view) as SharePoint, FIM, SQL Server, and AD do their magic dance. A successful run includes ten operations in my dev environment and I've previously posted a screen shot of this if you're interested.

If you look carefully, the operations view will reveal the user name involved with each operation and list some partition info as well. To dive in deeper, click the Management Agents button in the top menu; in my case, I'm presented with three MAs (if you've got more because you've been struggling with connections, you may be in trouble):

• The first MA named ILMMA connects to the database I specified when setting up UPS ("Sync DB")
• The second MA named MOSS-{GUID} connects to the ProfileImportExportService web service
• The final MA named MOSSAD-{name of my connection as configured in CA} connects to Active Directory

By viewing the properties for each MA (right-click on an MA and select Properties from the context menu or use the Actions pane to the right of the window) I can also examine specific properties to determine exactly what domain name FIM is configured to use and the accounts used to interact with AD, SQL Server, and the web service:

The attentive reader will note there's a lot of farm account action going on here and that's because both FIM services are configured to log on as the farm account and my understanding is they have to be because of the way the relevant timer job(s), which are also run as the farm account, interact with these services (says Spence). I'll also point out my svc_spups account is the account to which I've granted Replicating Directory Changes in AD.

If you found this post helpful, please support my advertisers.

Testing database connectivity

Developer that I am, I've gone so far as to write throwaway console apps to test connectivity to a database server as a different user; hopefully I'll never have to do that again after learning about this really cool trick:

• Create a new text file and change the extension to .udl
• Double-click it
• Select a provider, configure the options to connect and hit Test Connection (note the Use a specific user name and password option means SQL authentication; entering an AD account here will fail).
• Start a console window as another user and execute the .udl file to connect as someone else using integrated security if necessary
• Having ruled a security issue in or out, fix the problem!

 

A massive shout out to Todd Klindt for sharing this in one of his recent netcasts!!!

Ps. "OK" through to persist the connection details in the form of a connection string to your text file. Kinda helpful!

If you found this post helpful, please support my advertisers.

miisclient.exe success output

Partly for the glory but more for posterity (my own future reference!), I've screen captured what a successful SharePoint 2010 AD user profile synchronisation looks like in the FIM client:

Will this gain me entry to a special club or something? If so, thanks Spence! Now to work through all the event log errors and warnings… perhaps I ought to call it a day!

If you found this post helpful, please support my advertisers.

Where to find the FIM Client (miisclient.exe)

Note to self and anyone else who cares… you'll find the FIM client (aka miisclient) at the following location:

C:\Program Files\Microsoft Office Servers\14.0\Synchronization Service\UIShell\miisclient.exe

Use it to debug SharePoint 2010 user profile synchronisation.