Fixing the Save site as a template error when you have provisioned custom site columns in SharePoint 2013

***UPDATE: 18/03/2015***
I’ve updated this article to discuss the specifics of when this issue can arise

I had a report of a user experiencing an error using the Save site as a Template on a straight forward Team Site in 2013.

tl;dr
If you provision a custom site column with the Overwrite=”true” attribute using SPFieldCollection.AddFieldAsXml then the Save site as template function will fail as the underlying code to generate the wsp file will add a duplicate Overwrite=”true” attribute to the site column and therefore generate an xml file error. You need to update any provisioned site column’s SchemaXml property to remove the Overwrite=”true” attribute.

I was able to re-create the error on a test server and began the investigation.

Note: the farm was patched up to August 2013 CU only, but I’ve had a quick check of the codebase in SP1 and it appears that this issue discussed below persists

In ULS I saw the following error for the correlation id:

[Forced due to logging gap, Original Level: Monitorable] System.Xml.XmlException: ‘Overwrite’ is a duplicate attribute name. Line 1, position 327.
at System.Xml.XmlTextReaderImpl.Throw(String res, String arg, Int32 lineNo, Int32 linePos)
at System.Xml.XmlTextReaderImpl.AttributeDuplCheck()
at System.Xml.XmlTextReaderImpl.ParseAttributes()
at System.Xml.XmlTextReaderImpl.ParseElement()
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at Microsoft.SharePoint.SPSolutionExporter.WriteXmlToWriter(XmlWriter output, String xml, Boolean skipDocumentElement, Boolean addCdata)
at Microsoft.SharePoint.SPSolutionExporter.ExportFields(SPFieldCollection fields, String partitionName)
at Microsoft.SharePoint.SPSolutionExporter.ExportListsManifest(ListInstancesExportSummaryInfo exportSummary, ModuleExportSummaryInfo moduleExportSummary, List`1 workflowContentTypes, String workflowForm, String serverRelativeworkflowForm)
at Microsoft.SharePoint.SPSolutionExporter.ExportLists()
at Microsoft.SharePoint.SPSolutionExporter.GenerateSolutionFiles()
at Microsoft.SharePoint.SPSolutionExporter.ExportWebAsSolution()

and

System.InvalidOperationException: Error generating solution files in temporary directory.
at Microsoft.SharePoint.SPSolutionExporter.ExportWebAsSolution()
at Microsoft.SharePoint.SPSolutionExporter.ExportWebToGallery(SPWeb web, String solutionFileName, String title, String description, ExportMode exportMode, Boolean includeContent, String workflowTemplateName, String destinationListUrl, Action`1 solutionPostProcessor, Boolean activateSolution)
at Microsoft.SharePoint.ApplicationPages.SaveAsTemplatePage.BtnSaveAsTemplate_Click(Object sender, EventArgs e)
at System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

The first clue here is that Overwrite is a duplicate attribute. And, that part of the stack trace:

at Microsoft.SharePoint.SPSolutionExporter.ExportFields(SPFieldCollection fields, String partitionName)

My first thought was were there any custom site columns on this site, and there were several. The custom site columns had been provisioned first to the farm Content Type Hub and then published to the site via a custom content type.

The custom site columns were deployed to the farm as part of a wsp, created in Visual Studio. The Visual Studio solution contained a few features that included your typical Elements.xml file containing some site columns defined using CAML.

Now if you’re like me you’ll “probably” have got into a habit of adding to your site columns the Overwrite=”TRUE” attribute, this is usually so that your develop, deploy, test workflow is continuously deploying the “most up to date” version of your custom artefacts. Also, it is a “requirement” for certain types of site columns to have Overwrite=”TRUE” as part of their definition.

As an example, the following are typical blog posts stating the requirement of adding the Overwrite=”TRUE” attribute:

https://mdashiqur.wordpress.com/2014/02/09/lookup-field-as-a-site-column-using-caml-sharepoint-lookup-field/

https://andrewonsoftware.wordpress.com/2011/10/27/how-to-define-lookup-column-via-caml-in-sharepoint-2010-and-avoid-errors/

We have established that our test site has custom columns that contain the Overwrite=”TRUE” attribute, but this seems to be causing an issue with Save site as a template.

Taking a step back into memory lane, what Save site as a template actually does is to create a wsp containing all the artefacts required to create a “copy” of a site. This can be very useful for Site Collection owners to define what each subsite should “look like”. Save site as a template is also a useful feature for developers to learn how to “write CAML”, and in fact what a lot of developers used to do would be to start off any custom site definition/web template by first prototyping in the UI, Save site as a template, download the wsp from the Site Collection solution gallery, and then crack open the wsp and add the files into Visual Studio.

The biggest issue with this prototyping approach was that the wsp generated by Save site as a template didn’t create “round-trippable” CAML definitions for fields. A developer would add all the wsp files into their solution, try to deploy it and it would “appear” to have worked but site columns such as Lookup columns wouldn’t get provisioned correctly. So “it is known” that using Save site as a template and Visual Studio together requires more tweaking of the generated files.

Now going back to the stack trace of the error in ULS I opened ILSpy and started to work my way through the code base for the SPSolutionExporter class, specifically on the Microsoft.SharePoint.SPSolutionExporter.ExportFields(SPFieldCollection fields, String partitionName) method.

Here’s what ILSpy decompiles the method to:

// Microsoft.SharePoint.SPSolutionExporter
private void ExportFields(SPFieldCollection fields, string partitionName)
{
	if (fields.Count <= 0 || this.WorkflowExportModeIsEnabled)
	{
		ULS.SendTraceTag(894035u, ULSCat.msoulscat_WSS_SolutionExporter, ULSTraceLevel.Verbose, "There are no fields to export for partition \"{0}\" so the feature will not be included in this solution.", new object[]
		{
			partitionName
		});
		return;
	}
	SPSolutionExporter.FieldsExportSummaryInfo fieldsExportSummaryInfo = new SPSolutionExporter.FieldsExportSummaryInfo();
	foreach (SPField sPField in fields)
	{
		string title = sPField.Title;
		try
		{
			SPSolutionExporter.FieldExportSummaryInfo fieldExportSummaryInfo = SPSolutionExporter.ExportField(sPField, this.web);
			fieldsExportSummaryInfo.FieldExportSummaryInfoEntries.Add(fieldExportSummaryInfo.SortedName, fieldExportSummaryInfo);
		}
		catch (Exception ex)
		{
			string strMessage = string.Format(this.web.UICulture, SPResource.GetString("SitePackaging_ErrorExportingField", new object[0]), new object[]
			{
				title
			});
			ULS.SendTraceTag(894036u, ULSCat.msoulscat_WSS_SolutionExporter, ULSTraceLevel.Monitorable, ex.ToString());
			throw new SPException(strMessage);
		}
	}
	string text = SPSolutionExporter.ConvertWebRelativeUrlToPartitionedRelativePath("ElementsFields.xml", partitionName);
	ULS.SendTraceTag(894037u, ULSCat.msoulscat_WSS_SolutionExporter, ULSTraceLevel.Verbose, "Creating field feature manifest file '{0}'", new object[]
	{
		text
	});
	using (ScopedXmlWriter scopedXmlWriter = new ScopedXmlWriter(this.CreateXmlWriterInStagingArea(text), text))
	{
		using (new ScopedXmlWriterElement(scopedXmlWriter.Value, "", "Elements", "http://schemas.microsoft.com/sharepoint/"))
		{
			foreach (KeyValuePair<string, SPSolutionExporter.FieldExportSummaryInfo> current in fieldsExportSummaryInfo.FieldExportSummaryInfoEntries)
			{
				SPSolutionExporter.FieldExportSummaryInfo value = current.Value;
				SPSolutionExporter.WriteXmlToWriter(scopedXmlWriter.Value, value.SchemaXml);
			}
		}
	}
	string text2 = SPSolutionExporter.ConvertWebRelativeUrlToPartitionedRelativePath("Feature.xml", partitionName);
	fieldsExportSummaryInfo.FeatureFileRelativePath = text2;
	ULS.SendTraceTag(894038u, ULSCat.msoulscat_WSS_SolutionExporter, ULSTraceLevel.Verbose, "Creating fields feature file '{0}'", new object[]
	{
		text2
	});
	using (ScopedXmlWriter scopedXmlWriter2 = new ScopedXmlWriter(this.CreateXmlWriterInStagingArea(text2), text2))
	{
		using (new ScopedXmlWriterElement(scopedXmlWriter2.Value, "", "Feature", "http://schemas.microsoft.com/sharepoint/"))
		{
			SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "Id", null, fieldsExportSummaryInfo.FeatureId);
			SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "Title", null, "Fields feature of exported web template \"" + this.web.Title + "\"");
			SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "Version", null, "1.0.0.0");
			SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "Scope", null, "Web");
			SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "Hidden", null, true);
			SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "RequireResources", null, true);
			using (new ScopedXmlWriterElement(scopedXmlWriter2.Value, string.Empty, "ElementManifests", null))
			{
				using (new ScopedXmlWriterElement(scopedXmlWriter2.Value, string.Empty, "ElementManifest", null))
				{
					SPSolutionExporter.WriteXmlAttribute(scopedXmlWriter2.Value, string.Empty, "Location", null, Path.GetFileName(text));
				}
			}
		}
	}
	ULS.SendTraceTag(894039u, ULSCat.msoulscat_WSS_SolutionExporter, ULSTraceLevel.Verbose, "Exported {0} fields into partition \"{1}\".", new object[]
	{
		fieldsExportSummaryInfo.FieldExportSummaryInfoEntries.Count,
		partitionName
	});
}

The key line in this is line 18:

		try
		{
			SPSolutionExporter.FieldExportSummaryInfo fieldExportSummaryInfo = SPSolutionExporter.ExportField(sPField, this.web);
			fieldsExportSummaryInfo.FieldExportSummaryInfoEntries.Add(fieldExportSummaryInfo.SortedName, fieldExportSummaryInfo);
		}

This is the line of code that generates the xml that is then eventually written to disk as an xml file. The SPSolutionExporter.ExportField method calls the SPSolutionExporter.GetFieldSchemaXml which in turn calls the SPSolutionExporter.GenerateSchemaXmlForExport method

internal string GenerateSchemaXmlForExport(bool addOverWriteAttribute, bool removeSealedAttribute)
{
	string text = this.SchemaXml;
	text = SPUtility.RemoveXmlAttributeWithNameFromFirstNode(text, "Field", "Version");
	if (removeSealedAttribute)
	{
		text = SPUtility.RemoveXmlAttributeWithNameFromFirstNode(text, "Field", "Sealed");
	}
	if (addOverWriteAttribute)
	{
		string attributeNameValue = "Overwrite" + "=\"TRUE\"";
		text = SPUtility.AddXmlAttributeToFirstNode(text, "Field", attributeNameValue);
	}
	return text;
}

As we can see from the above the snippet:

	if (addOverWriteAttribute)
	{
		string attributeNameValue = "Overwrite" + "=\"TRUE\"";
		text = SPUtility.AddXmlAttributeToFirstNode(text, "Field", attributeNameValue);
	}
}

The culprit is lines 11 and 12. The Overwrite=”TRUE” attribute is added to each field’s xml EVEN IF IT ALREADY EXISTS. I say that again, even if the current field’s SchemaXml property already contains the Overwrite=”TRUE” attribute, the SPSolutionExporter.GenerateSchemaXmlForExport adds it in again. And this is the reason why the error is being generated, we have custom site columns that contain the Overwrite=”TRUE” attribute and it is these custom site columns that are causing the Save site as a template functionality to fail.

Once this code unwinds eventually an attempt will be made to write an xml document out, and the string passed to the XmlWriter contains an element with two Overwrite=”TRUE” attributes and therefore fails.

If we think about what is happening here, what “I think” is that Microsoft have received feedback that the wsp file created using Save site as a template did not add in the correct “round-trippable” xml on certain types of columns, and so they have added in this code to the SPSolutionExporter class to mitigate against the scenario of a developer using the UI to prototype, save as a template, and add the wsp files into Visual Studio.

Unfortunately is seems that the very people they’re trying to help – developers – are most likely going to be impacted by this bug as it is developers who will be creating custom site columns in CAML (and migrations from 2010 will almost certainly have custom site columns defined via CAML).

***UPDATE: 18/03/2015***
I’ve identified that the cause of this issue is when custom columns are added using the SPFieldCollection.AddFieldAsXml method and not as a declaritive field via a feature. The codebase for SPFieldCollection.AddFieldAsXml must be taking the xml passed to it and copying it to the SPField.SchemaXml property.

Now that we have identified the cause, is there a solution to this issue? The first solution would be to never deploy any site columns that are defined via CAML and contain the Overwrite=”TRUE” attribute.

But that’s not perhaps a pratical solution, there are already many site columns defined in this way, and it might not be feasible to re-deploy these site columns (in particular, a lot of farms will have deployed custom content types to the content type hub, and the admins of these farms may well have scenarios where they cannot re-publish the content types).

Also, (and I’ve not tested this) it might not be possible to get custom site columns deployed if they are defined with CAML and they are of type lookup.

Going back to the codbase of SPSolutionExporter, SPField.SchemaXml property is what is used to build up the xml that will eventually be written to disk:

internal string GenerateSchemaXmlForExport(bool addOverWriteAttribute, bool removeSealedAttribute)
{
	string text = this.SchemaXml;
	text = SPUtility.RemoveXmlAttributeWithNameFromFirstNode(text, "Field", "Version");

So is there a way for us to (1) Identify the fields that are causing the error and (2) update the fields to remove the error. The answers are yes and yes. The following Powershell snippet can be used to identify fields that will cause the error:

$w = Get-SPWeb -Identity https://test.company.internal/sites/testsite
$w.Fields | ?{$_.SchemaXml -like "*Overwrite=*"} | select Title, StaticName | ft

And the following powershell can be used to update the SchemaXml property to “remove” the issue:

$w = Get-SPWeb -Identity https://test.company.internal/sites/testsite
$fs = $w.Fields | ?{$_.SchemaXml -like "*Overwrite=*"}
foreach($f in $fs){
    $f.Title
    $f.SchemaXml = $f.SchemaXml.replace("Overwrite=`"TRUE`"", "")
}

You can run this on an individual site to allow you to save the site as a template. If you have a content type hub, then run the snippet against the content type hub, but then (importantly) make sure you publish all content types that use the custom site columns.

YMMV

 

 

 

 

 

 

 

 

Advertisements
This entry was posted in CSI SharePoint. Bookmark the permalink.

2 Responses to Fixing the Save site as a template error when you have provisioned custom site columns in SharePoint 2013

  1. Cool, realy appreciated, i had same conclusion and was working on the powershell part today when i have stumbled on yours. Fixed for me, just Overwrite was = FALSE. I have removed it but just wondering if it will come back after solution upgrade.. most likely..

    PS.: i was working on sp2010

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s