Errors while deploying sandboxed solutions

There are lots of posts out there dealing with the various troubles you can run into when trying to get your sandboxed solutions to work as they should. Most focus on getting the User Code Host to start and they do a pretty good job explaining that.

However, there is one scenario that doesn't appear to get as much attention as it should; what happens when you don't use the farm account for the sandbox service. By default, SharePoint uses the farm account for the User Code Service. However, Microsoft advises us wisely to use separate accounts for separate services. This allow for a better management of privileges. Never is this requirement more clear then when you are configuring a sandbox. Using the farm account, which has a lot of privileges, partially defeats the purpose of having sandbox has in the first place.

So, we see more and more production servers with separate accounts for the user code service, which is a good thing. However, developers are also exposed to this and get errors they don't understand:

When deploying by hand you might see the following in the ULS:

Failed to load receiver assembly "-------------, Version=1.0.0.0, Culture=neutral, PublicKeyToken=-----------------" for feature "-----------------" (ID: ------------------------------).: System.IO.FileNotFoundException: Could not load file or assembly '-------------, Version=1.0.0.0, Culture=neutral, PublicKeyToken=-----------------' or one of its dependencies. The system cannot find the file specified.  File name: '-------------, Version=1.0.0.0, Culture=neutral, PublicKeyToken=-----------------'    
 at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)    
 at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)    
 at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)    
 at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)    
 at System.Reflection.Assembly.Load(String assemblyString)    

Or when deploying from visual studio:

Error occurred in deployment step 'Add Solution': Unable to load assembly group. The user assembly group provider threw an exception while trying to provide user assemblies for the specified assembly group.

I wrote this up because I found a lot of posts online which advised you to change the service account for the sandbox service (by hand, through services.msc no less!). Don't do this. The solution is the easiest part of this post: Grant the sandbox service account full control on the webapplication(s) you wish to deploy your sandboxed solutions to. No reboots, recycles or any wait required.

Fixing folders that have been converted into document sets

SharePoint 2010 saw the introduction of 'Document Sets'. In my opinion these are the greatest improvement in document management since the stapler. The concept is very simple; you create sets of documents. These sets have their own introduction page, where you can leave relevant info. Also, the documents in the set can share metadata properties and the set itself can have distinct properties which may or may not get pushed down to the documents in the set.

Under the hood, these document sets are nothing more than the old fashioned folders on steroids. This becomes clear when you change the content type for a folder. If you have enabled the document sets site feature, and added the document set content type to your library, you can change folders into document sets. This is especially useful in migration scenario's, if you were already using folders for the same purposes you now would like to use document sets for.

However, the result of this content type switch is not the same as when you create a new document set. Some properties do not update correctly. The resulting document set does not get its landing page and it doesn't get the pretty icon. The culprit is the ProgId property, which should be set to 'SharePoint.DocumentSet'. This property is used by SharePoint to launch the correct program when you open a document (such as a Word or Excel document).

You can go one of two ways in fixing an issue like this. You can create an event receiver which sets the property correctly when the change is made. This is the approach used in a project on codeplex, SharePoint 2010 Folder To Document Set Conversion Fix, by Robert R. Freeman. The main advantage of this solution is that it will fix the document set the moment you update it. A drawback is that you need to deploy custom code, which isn't always an option. Also, folders that have had their content type updated in the past are not fixed

A second solution is to just script it, and fix the document sets that are not functioning correctly in one go. For this you can use the script below.

Of course, all the regular legal stuff applies. Use at your own risk. I do not accept any liability for what happens when you run this script. And never, never, run a random script you've downloaded from the internet against a production environment without properly testing and examining it.

param(
 [string]$WebUrl = $(throw "WebUrl required."),
    [string]$ListName = $(throw "ListName required."),
    [bool]$disableEventFiring = $false
)

#Region [ Load Assemblies ]
$spNotFoundMsg = "SharePoint not found. Run this script from a SharePoint server that is part of the farm where you want to update your content types"

if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") -eq $null)     { throw $spNotFoundMsg; }
if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server") -eq $null)    { throw $spNotFoundMsg; }
if ([Reflection.Assembly]::LoadWithPartialName("System.Xml.Linq") -eq $null)      { throw $spNotFoundMsg; }
if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server") -eq $null)    { throw $spNotFoundMsg; }
if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.UserProfiles") -eq $null) { throw $spNotFoundMsg; }
if ([Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Taxonomy") -eq $null)   { throw $spNotFoundMsg; }

$snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
if ($snapin -eq $null)
{
   Write-Output "Loading Microsoft SharePoint Powershell Snapin"
   Add-PSSnapin "Microsoft.SharePoint.Powershell"
} 
#endregion 


Start-SPAssignment -Global

$web = Get-SPWeb $WebUrl -EA 1 

$list = $web.Lists[$ListName]
if ($list -eq $null) { throw "List '$ListName' not found at '$WebUrl'" }

Write-Host "Found list '$ListName' at '$WebUrl'" -ForeGroundColor Green


if($disableEventFiring)
{
    Write-Host "Disabling event firing" -ForeGroundColor Yellow
    $myAss = [Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint");
    $type = $myAss.GetType("Microsoft.SharePoint.SPEventManager");
    $prop = $type.GetProperty([string]"EventFiringDisabled",[System.Reflection.BindingFlags] ([System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Static));
    $prop.SetValue($null, $true, $null);
}

$count = 0
$folderCollection = $list.Folders
foreach($folder in $folderCollection)
{
    if ($folder.ContentType.Id.ToString().StartsWith("0x0120D520"))
    {   
        if($folder.ProgId -ne "SharePoint.DocumentSet")
        {
            Write-Host "Found a document set to fix:" $folder.Title -ForeGroundColor Yellow
            
            $folder.ProgId = "SharePoint.DocumentSet"
            $folder.Update()
            $documentSet = [Microsoft.Office.DocumentManagement.DocumentSets.DocumentSet]::GetDocumentSet($folder.Folder)
            $documentSet.Provision()
            
            Write-Host "Document set" $folder.Title "is fixed" -ForeGroundColor Green
            $count++
        }
    }
}

if ($count -eq 0)
{
    Write-Host "Found no document sets requiring an update!" -ForeGroundColor Yellow
}
else
{
    Write-Host "Updated $count document sets!" -ForeGroundColor Green
}

Stop-SPAssignment -Global


<#
.SYNOPSIS
Fix document sets in a SharePoint library

.DESCRIPTION
When manually updating the content type of a folder to a document set, not
all properties are propertly updated by SharePoint. This script fixes these
broken document sets. This script can be run more than once.

.PARAMETER WebUrl
Specifies the url the web hosting the document sets to fix

.PARAMETER ListName
Specifies the display name of the library holding the document sets 
to fix

.PARAMETER disableEventFiring
Optional parameter. Set to $True to prevent the update from firing 
events for event receivers. Can be useful in custom code scenario's

.INPUTS
None. You cannot pipe objects to DocumentSetFixer.ps1.

.OUTPUTS
None. DocumentSetFixer.ps1 does not generate any output.

.EXAMPLE
C:\PS> .\DocumentSetFixer.ps1 -WebUrl "http://mysitecollection/myweb" -ListName "Shared Documents"
Fix all documents sets in the library "Shared Documents" on the 
SharePoint web at "http://mysitecollection/myweb".

.EXAMPLE
C:\PS> .\DocumentSetFixer.ps1 -WebUrl "http://mysitecollection/myweb" -ListName "Shared Documents" -disableEventFiring $true
Fix all documents sets in the library "Shared Documents" on the 
SharePoint web at "http://mysitecollection/myweb". This fix will not 
trigger event receivers on the document sets.

.LINK
Developed by: http://www.vxcompany.com

#>

"Authenticated users" and the taxonomy hidden list

When provisioning a new SharePoint site, a few groups are automatically created for you. With most templates you automatically get a visitors, members and owners group. On publishing sites and record centers you get a lot more.

For a new Team Site, the "All Groups" page initially looks like this:

5 groups, most of them are pretty obvious. But what is this "Authenticated Users" group doing here? Basically, this is a group that is available through IIS and contains every user that can be authenticated by IIS, i.e., have their identity verified. For instance, all AD users automatically fall into this category.

From TechNet:

SID: S-1-5-11
Name: Authenticated Users
Description: A group that includes all users whose identities were authenticated when they logged on. Membership is controlled by the operating system.

If you want to know what permissions a group has within your site, SharePoint 2010 offers us the very useful "Check Permissions" option in the Site Permissions screen:

This shows us the Authenticated Users haven't got any permissions on our site.

So... an over-active user with spring cleaning in mind, could be forgiven for deleting this group from the site collection. This is pretty easy and you only get a very nondescript warning message:

However, the permission listing does not take permissions on hidden lists in account. Specifically the Taxonomy Hidden List (which is used for caching fields from the managed metadata service application), has some interesting permissions:

(Reach this list at [site]/lists/taxonomyhiddenlist)

Removing the Authenticated Users from your site will also remove their permissions from this list. It will cause strange behavior in your site. List views with managed metadata columns will suddenly have blank values. Edit and view forms for list items may show unauthorized exceptions. However, for site collection administrators everything will continue to work, so the problem may not be directly obvious. And when the action and the moment problems are reported lay further apart, trouble shooting becomes a challenge.


Bottom line: never remove the SharePoint groups that have been created for you by features or deployment

Further reading: A good post on the inner workings of the Taxonomy Hidden List


Update: The same problem can occur with the masterpage gallery. The publishing feature sets some limited reading rights on the gallery for all authenticated users. Removing these permissions will result in access denied on edit mode for pages for all users, including associated owners. Only site collection admins will still be able to edit pages.

Changing the order of fields in edit or display forms

Just a quick post in answer to a question I got yesterday:

How do you change the order of fields in edit or display forms?

It's a question a lot of end users struggle with, and I never know the answer without Google. So, I thought I'd put it up here, with some more relevant keywords, so others and I can find it more easily in the future.

1. Go to the list
2. Enter list settings (from the ribbon in 2010, from the drop downs in 2007)
3. Click Advanced settings
4. Ensure ‘allow management of content types’ is checked
5. Go back to the list settings
6. In the list of content types associated with the list, click the one you want to change the order of fields for (in lists that have been created ad hoc this is usually item or document).
7. In the bottom of the screen a link appears called ‘Column order’

Hope it helps someone.

Deploying solutions to a specific Web Application

SharePoint solutions (non-sandboxed) can be deployed at two scopes;

• Globally
• Web Application

SharePoint has the annoying habit of forcing you to deploy globally whenever possible. When you try to deploy a global solution to a specific url you get the following message:
- This solution contains no resources scoped for a Web application and cannot be deployed to a particular Web application.

In almost all cases I want to deploy my solution to a single web application. This makes creating reusable deployment scripts much easier. It also adds logic to your deployment scenario's and farm solution overview in Central Administration.

Update
Another important reason to deploy to a single web application is that upon deployment, retraction or updating of your solution you can avoid restarting all application pools. This limits downtime, especially when you are not alone in your farm:

Avoid creating a lot of global SharePoint Packages and try instead to provision as much as you can to specific Web Applications. Every time you touch a global SharePoint Package all Applications Pools will be stopped/recycled. Although there are some scenarios when you can’t avoid creating global SharePoint Packages, you should try to avoid them

An interesting post by Waldek on the subject.
/Update

The trick is fooling SharePoint into registering your solution as a web application scoped solution. SharePoint checks wether there are items which have to be deployed to a specific web application when your solution is added to the solution gallery. One of the elements that SharePoint checks for are safe control entries. These have to be merged into a web.config for a specific web application. The easiest way to force deployment to a specific web application is adding a dummy safe control entry to your package.

Here is how you do that:



1. Double click the package
2. Open the manifest
3. Edit the options
4. Add you dummy data. In my solution I added the following:

<Solution xmlns="http://schemas.microsoft.com/sharepoint/"> 
 <Assemblies>   
  <Assembly Location="SharePointProject1.dll" DeploymentTarget="GlobalAssemblyCache">     
   <SafeControls>       
   <SafeControl Assembly="SharePointProject1,Version=1.0.0.0, Culture=neutral, PublicKeyToken=****************" Namespace="SharePointProject1" TypeName="*" />     
   </SafeControls>   
  </Assembly> 
 </Assemblies>
</Solution>

That's all there is to it. This solution will now only deploy at the web application scope!


Note: For MOSS this works the same. How to add the entry is a bit different depending on your wsp packaging tool. With STSDEV you can add the entry to the SolutionConfig.xml.

Exceptions when creating site columns based on local term sets

This blog is about a little known bug in SharePoint 2010. The bug manifests as follows:

When creating a new site column of the managed meta data type, and you select "customize your term set", you are presented with one of the following errors:
- This operation cannot be completed. The term store may be unavailable.
- Failed to read from or write to database. Refresh and try again. If the problem persists, please contact the administrator






Reason

When you create a site column, the first field is the title. This field is required, but you can start filling out the rest of the form before entering the title. When you choose "Managed Meta data", and then "Customize your term set", an empty term set is directly created for you. This term set is titled "Untitled". This term set is stored in the managed meta data service. Even when you rename the term set afterward, and give the site column a proper name, a reference to this "Untitled" site column is kept, linked to the url of your site collection. You can see this when you create a new column, and again don't specify a title. The suggested title for your term set will be "Untitled_1".

As long as you keep your site collection, this is not a problem. However, when you delete this site collection and create a new site collection at the same url, this reference causes problems. Because it was stored in the service application, it was not deleted.

It is always in the way when you try to create new site columns with customized term sets.


Solution
The only solution I found is creating a new Managed Meta data Service instance. You might want to delete the old one, with the corrupt data, but based on business needs this may not be an option. Remember to configure the new Managed Meta data Service instance with "This service application is the default storage location for column specific term sets" set to true:





If you don't, you'll get the following error:
"The default term store for this site cannot be identified"

Update: The first service pack for SharePoint contains a fix for the following problem:
- If a user creates a site collection, deletes it, and then re-creates the site collection by using the same name, the site collection group is not re-created in the term store.
If I'm not mistaken, this is a fix for the bug described here.

Usefull forms in the Layouts directory

It's been a while since I last posted here. Almost 2,5 years have passed since my last post. I've still been doing SharePoint, I just never got around to blogging about it. I'll try to post more in the coming months, so check back for updates.


To kick things of, I have a little tip that can be very useful to site administrators. SharePoint has a lot of forms for managing the site. These are located in the 14 hive's layouts directory (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS).

Some of these are available through the site settings menu. Others are only seen when SharePoint redirects you to them. Most of these however, are also available by just browsing to them. I'll discuss 2 examples, but many more usefull forms are available.

WARNING
The forms mentioned in this page are hidden for a reason. Some of these may not work in all circumstances. Others, such as the 'Save site as template' let you perform actions that are not officially supported. Always use common sense when using these things and try the effects in a development or testing environment if at all possible.
/WARNING


In site without publishing features you cannot change the masterpage. As you can see here, there is no 'Masterpage' link:



Most of us SharePoint-devers are pretty codeminded, and will choose to just change the masterpage through code, or in our site/web template. However the form is still available:




All you need to do is browse to the /_layouts/ChangeMasterPage.aspx page.


A second example is the PermSetup.aspx page. This is the page you get when you choose to use unique permissions for your site. It allows you to manually set the associated member and owner groups for your site. This screen is quite important, because it is the only chance you have for setting the association through the UI. However, the only time you get there is at site creation. Unless you just browse to the /_layouts/PermSetup.aspx page:




Hope these two will save you the trouble of writing custom code.

Small update, the link to save your site as a template isn't shown when publishing is enabled. This doesn't mean it's gone:
/_layouts/savetmpl.aspx