Justin Kobel's Blog

Microsoft Collaboration (SharePoint, OneDrive, and Other Related Technologies)

Make a SharePoint Online Page Show Up on "News" Web Parts

When you click the "New" button on a News web part, it invokes a new page interface within SharePoint Online, and you start writing your page.  Once published, it shows up on the "News" web part on the home page.  Pretty easy, right?

However, what happens if somebody has written a new page, and then after the fact asks you to add it to the "News" web part?  

To get this to work, you need to set a value on the Site Pages list that unfortunately (as of November 2017) you can't update via the UI.  You can get the "Promoted State" column to show up on Modern Views of the Site Pages library; but when you try to edit it (via Quick Edit or via the List details page), it doesn't show up at all. Based off some quick testing, it looks like if Promoted State is set to "2", it causes it to be treated as a News Article.  If Promoted State is any value other than "2", it doesn't show up.   

Luckily, our good friend JSOM, and even better, the PnP JS Library gives us an easy way to do this.  All you need is to get the list item ID of the page, and you can run the below.  You'll obviously want to update the ID 22 to the actual item ID of the page(s) in your Site Pages library, and you should be good to go.  

import pnp from "pnp";

pnp.sp.web.lists.getByTitle("Site Pages").items.getById(22).fieldValuesForEdit.get().then(function(data) {
    for(var k in data) {
        console.log(k + ": " + data[k]);
    }
});

pnp.sp.web.lists.getByTitle("Site Pages").items.getById(22).update({ PromotedState: 2})

Developing Your First SharePoint and Office Add-Ins

A big thanks goes out to all of the guys in the Nashville SharePoint Users Group for having me out this month to discuss developing your first SharePoint and Office Add-Ins.

As I noted during the presentation, I have uploaded the presentation and all of the code "snippets" I used throughout the demonstrations to my OneDrive.  Please reach out if you have any questions at all, or if you would like a full project file for the demonstrations.

 

Enabling Project Server Features on SharePoint 2016

Pi Day 2016 is now officially Sharepoint 2016 RTM day, at least for us SharePoint nerds… And those of us on KiZAN Technologies; who are now officially a Microsoft Project and Portfolio Management Partner, this means that Project Server 2016 is now available on-premises, and is now an integrated installation as a part of SharePoint 2016. 

As far as I'm aware, pending Project Server license keys being provided on the Volume Licensing Center in May 2016, Project Server licenses are still separate and additive to SharePoint Enterprise features; and I have heard no plans to the contrary. So even though Project Server 2016 functionality is included in the SharePoint 2016 installation media; its license is still separate.   

If you install SharePoint 2016 and try to create a Project Server based site collection, you’ll probably wind up with an error message stating:

The farm does not have a product key for Project Server. You can add your key by using Enable-ProjectServerLicense
The farm does not have a product key for Project Server. You can add your key by using Enable-ProjectServerLicense

Below is the step by step process to activate Project Server 2016 in a SharePoint 2016 environment.  Note that you will need a Project Server key in order to perform this process.  This is not the same as the SharePoint 2016 RTM trial key.  You will need to grab a Project Server 2016 key from your MSDN, TechNet, or a trial key for this.  At the time of writing this blog, there are some keys available at https://social.technet.microsoft.com/Forums/en-US/3bbdb4b0-7240-4d92-90bd-cb78fccd437f/project-server-2016-preview-requires-the-farm-to-have-a-valid-sharepoint-server-enterprise-product?forum=SP2016, but I cannot vouch as towards their source, or if they are trial or otherwise restricted.  This process, unlike most SharePoint processes when it comes to licenses, are a one-way-trip, so I would not recommend performing this without ensuring your organization is properly licensed for SharePoint 2016 Enterprise, Project Server 2016, and all of the various other components necessary.  If you ever need to disable this license, you can execute a “Disable-ProjectServerLicense” cmdlet to back out this functionality on your farm.

Enabling the Project Web App License

You will need to grab a Project Server 2016 key from your MSDN, TechNet, or a trial key for this.  At the time of writing this blog, there are some keys available at https://social.technet.microsoft.com/Forums/en-US/3bbdb4b0-7240-4d92-90bd-cb78fccd437f/project-server-2016-preview-requires-the-farm-to-have-a-valid-sharepoint-server-enterprise-product?forum=SP2016, but I cannot vouch as towards their source, or if they are trial or otherwise restricted.  This process, unlike most SharePoint processes when it comes to licenses, are a one-way-trip, so I would not recommend performing this without ensuring your organization is properly licensed for SharePoint 2016 Enterprise, Project Server 2016, and all of the various other components necessary.  If you ever need to disable this license, you can execute a “Disable-ProjectServerLicense” cmdlet to back out this functionality on your farm.

From  UAC’ed, SharePoint Administration PowerShell context, execute the following PowerShell:

Enable-ProjectServerLicense –Key “insert the key here”

Creating the Project Server Service Application and Starting the Service

If you want to do things the easy way, you can provision this Service Application through Central Administration.

  1. Click on “Application Management” and then “Manage Service Applications”
  2. Click “New” and select “Project Server Service Application”
    image
  3. Wait a few minutes, and your service application will display on your service application page. 
  4. Click on “System Settings” and “Manage Services on Server”
  5. Click “Start” next to the “Project Server Service” on the servers you wish to run this service.  In a production instance, you will normally want this running on “batch” or “application” servers, in a redundant fashion.

If you prefer to utilize a PowerShell, you can create the Service Application through a scripted process as well:

$appPool = Get-SPServiceApplicationPool "SharePoint Web Services" #insert your standard Service Application app pool host name here
$pssa = New-SPProjectServiceApplication -Name "Project Server Service Application" -ApplicationPool $appPool –Proxy
Get-SPServiceInstance -Server "servernamehere" | ? {$_.TypeName -eq "Project Server Application Service"} | Start-SPServiceInstance

You’ll notice that when you click inside of this service application, there isn’t much to worry about here. You’ll essentially just see a view of Project Web Apps’' configured in the farm.

image

Creating the Project Web App Site Collection

Once you have the service application configured, you will now need to create a location for the Project Web App to be hosted.  Just like in prior versions of Project Server, you can create as many Project Web App site collections as necessary in your farm (although  I’m sure there’s some application boundary out there somewhere…).  I would generally recommend creating PWA on a PWA-specific site collection; this is not something you want to clutter up a standard team site or intranet site collection with.

Project Server 2016 appears to not display the normal “Project Web App” site collection template through the UI even after activating the license, so you must provision the site collection via PowerShell with the New-SPSite cmdlet.  The PWA site collection template is “pwa#0”:

New-SPSite -ContentDatabase “WSS_Content_PWA” –Template “pwa#0” -Name “Project Web App” -OwnerAlias “domain\username”

Once created, you’ll want to enable the below site collection features either through the UI or through PowerShell as you may prefer:

Site Collection Features

  1. SharePoint Server Enterprise Site Collection features (8581a8a7-cf16-4770-ac54-260265ddb0b2)
  2. Project Web App Settings (697c64b9-3dff-4981-9394-0a62632120ec)
  3. Project Web App Ribbon (1d253548-c70d-40fd-9930-9d313bedc359) – this should be automatically triggered in the prior feature, but will sometimes fail
  4. Project Web App Approval Content Type (ad739f9e-1525-4dec-a25e-10821ca70c95) - this should be automatically triggered in the prior feature, but will sometimes fail.

More often than not, you’re going to want to enable Project Server Permissions mode, which allows for Project Server 2016 to better match a PMO-language security model rather than requiring SharePoint based permissions. If that’s the case, you will want to run this PowerShell:

Set-SPProjectPermissionMode -Url “insert pwa site collection URL here” -Mode ProjectServer

 

And there you have it, Project Server is now up and running in your environment!  Isn’t it beautiful? Now come the fun part, of actually getting it working.  I’d recommend starting off with finalizing a permission model and configuring the Resource import process; but that’s a post for another day…

image

Common Errors and Troubleshooting

Issue 1:

When Activating a site collection level feature, a ULS error will be logged with the following:

[bucketHash:DC81E9B8] Exception while running Exec Service. Service Name: 'ProjectQueueService16', Exec Path: 'C:\Program Files\Microsoft Office Servers\16.0\Bin\Microsoft.Office.Project.Server.Queuing.exe', Exception: 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations.' Exception details follow. Type: 'System.ServiceModel.AddressAlreadyInUseException', message: 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host w...    
Monitorable    ...ith the same IP endpoint but with incompatible binding configurations.'  and stack trace:    at System.ServiceModel.Channels.SocketConnectionListener.Listen()     at System.ServiceModel.Channels.ExclusiveTcpTransportManager.OnOpen()     at System.ServiceModel.Channels.TransportManager.Open(TransportChannelListener channelListener)     at System.ServiceModel.Channels.TransportManagerContainer.Open(SelectTransportManagersCallback selectTransportManagerCallback)     at System.ServiceModel.Channels.TransportChannelListener.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.ConnectionOrientedTransportChannelListener.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.TcpChannelListener`2.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(Tim...    
Monitorable    ...eSpan timeout)     at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at Microsoft.Office.Project.Server.Services.QueueExecService..ctor(String serviceName, Guid sspGuid, String namedEvent, Int32 parentProcessId)     at Microsoft.Office.Project.Server.Services.QueueServiceInfo.GetExecServiceToRun(String serviceName, Guid sspGuid, String namedEvent, Int32 parentProcessId)     at Microsoft.Office.Project.Server.Services.ServiceProgram.Run(String[] args) Exception while running Exec Service. Service Name: 'ProjectQueueService16', Exec Path:'C:\Program Files\Microsoft Office Servers\16.0\Bin\Microsoft.Office.Project.Server.Queuing.exe', Exception: 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations.'    
Exception while running Exec Service. Service Name: 'ProjectQueueService16', Exec Path: 'C:\Program Files\Microsoft Office Servers\16.0\Bin\Microsoft.Office.Project.Server.Queuing.exe', Exception: 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations.'    
[bucketHash:0064CAE8] [SERVICE] 'ProjectQueueService16': Exception during AppDomainCompleteCallback for App Domain 'ProjectQueueService16 Project Server Service Application 2 882cf640-b2cc-4b7d-a922-c0a16f66b10e', exception 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations.' Exception details follow. Type: 'System.ServiceModel.AddressAlreadyInUseException', message: 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service h...    
Monitorable    ...ost with the same IP endpoint but with incompatible binding configurations.'  and stack trace:   Server stack trace:      at System.ServiceModel.Channels.SocketConnectionListener.Listen()     at System.ServiceModel.Channels.ExclusiveTcpTransportManager.OnOpen()     at System.ServiceModel.Channels.TransportManager.Open(TransportChannelListener channelListener)     at System.ServiceModel.Channels.TransportManagerContainer.Open(SelectTransportManagersCallback selectTransportManagerCallback)     at System.ServiceModel.Channels.TransportChannelListener.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.ConnectionOrientedTransportChannelListener.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.TcpChannelListener`2.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at Microsoft.Office.Project.Server.Services.QueueExecService..ctor(String serviceName, Guid sspGuid, String namedEvent, Int32 parentProcessId)     at Microsoft.Office.Project.Server.Services.QueueServiceInfo.GetExecServiceToRun(String serviceName, Guid sspGuid, String namedEvent, Int32 parentProcessId)     at Microsoft.Office.Project.Server.Services.ServiceProgram.Run(String[] args)     at Microsoft.Office.Project.Server.Services.QueueService.Main(String[] args)     at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)     at System.AppDomain.ExecuteAssembly(String assemblyFile, String[] args)     at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)     at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.Office.Project.Server.Services.ExecProcess.AsyncExecuteAssembly.EndInvoke(IAsyncResult result)     at Microsoft.Office.Project.Server.Services.ExecProcess.AppDomainCompleteCallback(IAsyncResult asyncResult) [SERVICE] 'ProjectQueueService16': Exception during AppDomainCompleteCallback for App Domain 'ProjectQueueService16 Project Server Service Application 2 882cf640-b2cc-4b7d-a922-c0a16f66b10e', exception 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations.'    
Error during Service Application Domain Callback. Service Name: 'ProjectQueueService16', AppDomain: 'ProjectQueueService16 Project Server Service Application 2 882cf640-b2cc-4b7d-a922-c0a16f66b10e', Exception: 'There is already a listener on IP endpoint 0.0.0.0:16002. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations.'    
[SERVICE] ProjectQueueService16: ProcessWatcher signaled    deb50447-52ea-e511-a380-000d3a713930
[SERVICE] ProjectQueueService16: ProcessWatcher restarting service for service app Project Server Service Application 2, id 882cf640-b2cc-4b7d-a922-c0a16f66b10e, service name ProjectQueueService16    deb50447-52ea-e511-a380-000d3a713930
The application domain ProjectQueueService16 Project Server Service Application 2 882cf640-b2cc-4b7d-a922-c0a16f66b10e is unloading and going to be recycled.    

Resolution 1:

Confirm you only have one Project Server Service Application bound to the proxy group you have on your web application.  You can have more than one Project Server Service Application, but there can only be one per web app proxy binding.

Issue 2:

Failed to get [Admin] access level connection string because current user is not an administrator on the farm (database name=[SP13RTM_WSS_Content], Id=f0061954-b15c-4870-b292-c0d6bb47ffd2, server name=spsql)    eda6689d-8b18-40bd-ac3a-fa8dd58dd367
Feature receiver assembly 'Microsoft.Office.Project.Server.Administration, Version=16.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c', class 'Microsoft.Office.Project.Server.Administration.ProvisionFeatureEventHandler', method 'FeatureActivated' for feature '697c64b9-3dff-4981-9394-0a62632120ec' threw an exception: System.InvalidOperationException: Only Administrator users can obtain the connection string for [Admin] access level     at Microsoft.Office.Project.Server.Administration.ProjectContentDatabase.GetConnectionString(DatabaseAccessLevel accessLevel)     at Microsoft.Office.Project.Server.DataAccessLayer.SqlSessionManager.GetSession(IProjectDataStorage projectDataStorage, Guid siteAffinity, DatabaseAccessLevel databaseAccessLevel, Int32 sqlCommandTimeoutSeconds)     at Micr...    eda6689d-8b18-40bd-ac3a-fa8dd58dd367
03/15/2016 03:11:43.10*    w3wp.exe (0x1A98)                           0x0D5C    SharePoint Foundation             Feature Infrastructure            88jm    High        ...osoft.Office.Project.Server.Administration.ProjectDatabaseConfiguration.Populate(IProjectDataStorage projectDataStorage, Int32 lcid, Guid siteId)     at Microsoft.Office.Project.Server.Administration.ProvisionFeatureEventHandler.FeatureActivated(SPFeatureReceiverProperties properties)     at Microsoft.SharePoint.SPFeature.DoActivationCallout(Boolean fActivate, Boolean fForce)    eda6689d-8b18-40bd-ac3a-fa8dd58dd367
Feature Activation: Threw an exception, attempting to roll back.  Feature 'PWASite' (ID: '697c64b9-3dff-4981-9394-0a62632120ec').  Exception: System.InvalidOperationException: Only Administrator users can obtain the connection string for [Admin] access level     at Microsoft.Office.Project.Server.Administration.ProjectContentDatabase.GetConnectionString(DatabaseAccessLevel accessLevel)     at Microsoft.Office.Project.Server.DataAccessLayer.SqlSessionManager.GetSession(IProjectDataStorage projectDataStorage, Guid siteAffinity, DatabaseAccessLevel databaseAccessLevel, Int32 sqlCommandTimeoutSeconds)     at Microsoft.Office.Project.Server.Administration.ProjectDatabaseConfiguration.Populate(IProjectDataStorage projectDataStorage, Int32 lcid, Guid siteId)     at Microsoft.Office.Project.Server...   

Resolution 2:

The account enabling this feature must be a “named” farm administrator.  Being a Local Administrator on the SharePoint Server(s), and having Local Administrators in the Farm Administrator group will fail if you are running these commands from a remote desktop or in a non-UAC’ed IE session while on the server.  Adding your user account directly to the Farm Administrators group will resolve this issue, or logging in as a Farm Account and executing the Feature Activation command from a UAC’ed PowerShell should resolve this issue:

SPS Cincinnati - SharePoint 2013 Farm Topology Planning

As promised to those who attended my session, here's my slides from today's presentation at SharePoint Saturday Cincinnati on SharePoint 2013 farm sizing and topologies (and a quick preview into SharePoint 2016). 
If anybody would like copies of my Permon and PAL templates used to analyze server performance, please reach out and let me know!


SharePoint 2013–Extract Usernames and Passwords from Secure Store Service

In a perfect world, we all have proper documentation in place and log our configuration of usernames and passwords within our Secure Store Service; but more often than not I know I have to guess at what username/password is actually in use within any given Application inside of the Secure Store Service. 

Below is some PowerShell that you can run (on the actual SharePoint server, there is no client-side access to do this as best as I can tell) in order to extract the usernames and passwords out of any AppId inside of the Secure Store Service application. The below GAC locations for the assemblies assume you’re on SharePoint 2013; but this can be repurposed for SharePoint 2010 by just pointing to the other GAC locations for the assemblies.


Add-PSSnapin 'Microsoft.SharePoint.Powershell'

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.dll"
Add-Type -Path "C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.Office.SecureStoreService\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.Office.SecureStoreService.dll"
Add-Type -Path "C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.BusinessData\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.BusinessData.dll"

$sssAppId = "ExcelHost"
$webAppUrl = "https://portal.kizan.com"  #Note - this can be any web application bound to the SSS proxy application; or central admin

$site = Get-SPSite -Identity $webAppUrl 
$ssProvider = New-Object("Microsoft.Office.SecureStoreService.Server.SecureStoreProvider")

$context = [Microsoft.SharePoint.SPServiceContext]::GetContext($site)
$ssProvider.Context = $context;

Write-Host "Credentials for SSS Application $sssAppId"
$credentialCollection = $ssProvider.GetCredentials($sssAppId)
foreach($c in $credentialCollection)
{
    $ptr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($c.Credential)

    $decryptString = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($ptr)

    Write-Host "Credential Type: " $c.CredentialType
    Write-Host "Decrypted String: " $decryptString
}

Business Critical OneNote-Part 3: Practical Recommendations

This is part 3 of my (frequently delayed) Business Critical OneNote Series, the one where I actually can give some recommendations rather than just adding on problems!

If you haven’t ready the previous two blogs, you can see them below:

Part 1 – Anatomy of a Notebook
Part 2 – The Sometimes On Recycle Bin?
Part 3 – Practical Recommendations (this post)

So, in summary, OneNote edits are not captured by SharePoint versioning, and deletes are captured in the OneNote 2013 client, but not when the deletions are caused by the OneNote Web App/OneNote Online.

As such, there are really two types of recommendations, those for on-premises users, and those in Office 365/Office Online.

Disable WOPI Bindings

Can be used for: On-Premises only

Since all of our deletions that don’t “recycle” occur only in the web app, we can just disable our WOPI binding specifically for editing OneNote files on our SharePoint servers.

From a SharePoint Management Shell window on any SharePoint server in your farm, just execute the following PowerShell:


New-SPWOPISuppressionSetting -Action "edit" -ProgId "OneNote.Notebook";
New-SPWOPISuppressionSetting -Action "editnew" -ProgId "OneNote.Notebook";
New-SPWOPISuppressionSetting -Action "editnew" -Extension "ONE";
New-SPWOPISuppressionSetting -Action "editnew" -Extension "ONETOC2";
New-SPWOPISuppressionSetting -Action "editnew" -Extension "ONEPKG";

This will basically make OneNote read-able in the web browser,  but all of the Edit buttons will only function to launch the OneNote desktop client.  Note that this method isn’t full proof, there’s various ways a user could try to “trick” SharePoint into letting them edit in the browser, this will basically remove the temptation to do so for anybody but us fellow SharePoint developers. 

Of course, though, this setting affects all of our web applications at once (at least all of our web applications using a single WOPI zone/AAM), so it’s a pretty broad brush we’re forced to use.

Setting the Library Default

Can be used for: On-Premises and SharePoint Online

If you can’t suppress OneNote web editing across the entire web application, you can at least make it less tempting for users.  In the document library that contains your OneNote notebook, click on the “Library” ribbon button and select “Library Settings”.

image

Select “Advanced Settings”

image

Select the option for “Opening Documents in the Browser” to “Open in the client application”
image

This will cause the default action for a user clicking on a Notebook to open up in OneNote, but they can still hover over it and select ‘'”Edit in Browser” should they opt to do so.

SQL Backup/Restore

Can be used for: On-premises only.

As all good on-premises administrators should, your SQL database should be backed up at least once a day, although I’m a big fan of incremental backups throughout the day at specific intervals if your SQL environment can sustain it.  As a starting point, I like to run them every 3 hours during normal operating hours, and then run a full after-hours (or at least once a week if you can’t sustain full backups each day).

In SharePoint 2010, we also got a new piece of functionality, the unattached database attach method.  This basically lets us selectively restore content from SharePoint content databases without having to use the whole “recovery farm” technique we did in SharePoint 2007 and earlier.  Below are the exact steps that can be used to rescue specific sections from a notebook that may have been deleted, updated, or otherwise in the Office Web Apps framework.  Of course, these backups are only as granular as your backup schedules, so if somebody made an edit and then deleted a page in between your backups, you’re out of luck, but hopefully that’s a rare circumstance.

The below script assumes the following:
The notebook is named “OneNote Test Notebook”
The web where it’s stored is: https://portal.contoso.com/OneNoteTest 
The site collection is stored in a content database: WSS_Content_Root
The site collection content database has been restored to: WSS_Content_Root_Restored


$dbServer = "SPSQL"    
$dbName = "WSS_Content_Root_Restored"     
$webUrl = "OneNoteTest"     
$listUrl = "/SiteAssets/OneNote Test Notebook"     
$localDirToExtractTo = "sptest13\Unattach\"
Add-PSSnapin Microsoft.SharePoint.Powershell     
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") 

$unattachedDb = Get-SPContentDatabase -ConnectAsUnattachedDatabase  -DatabaseName $dbName -DatabaseServer $dbServer
#since the unattached DB is basically mounted to central admin, we will go ahead and get it's URL    
$CAWeb = Get-SPWebApplication -IncludeCentralAdministration | where {$_.IsAdministrationWebApplication}    
$CAURL = $CAWeb.Url

 

$NotebookToExtract = New-Object Microsoft.SharePoint.Deployment.SPExportObject $NotebookToExtract.Type = [Microsoft.SharePoint.Deployment.SPDeploymentObjectType]::Folder  #remember, OneNote's just a glorified folder in SP $NotebookToExtract.Url =  $webUrl + $listUrl $NotebookToExtract.IncludeDescendants = [Microsoft.SharePoint.Deployment.SPIncludeDescendants]::All $ExportSettings = New-Object Microsoft.SharePoint.Deployment.SPExportSettings $ExportSettings.UnattachedContentDatabase = $unattachedDb $ExportSettings.SiteUrl = $CAURL $ExportSettings.FileLocation = $localDirToExtractTo $ExportSettings.LogFilePath = $localDirToExtractTo $ExportSettings.BaseFileName = "Notebook.cmp" $ExportSettings.ExportObjects.Add($NotebookToExtract) $ExportSettings.IncludeVersions = [Microsoft.SharePoint.Deployment.SPIncludeVersions]::CurrentVersion $ExportSettings.ExportMethod = [Microsoft.SharePoint.Deployment.SPExportMethodType]::ExportAll $ExportJob = New-Object Microsoft.SharePoint.Deployment.SPExport($ExportSettings) Write-Host "About to run the export job" $ExportJob.Run()

After running this script, you should have a .cmp of the extract sitting in your export directory noted above.  If you wanted to, you could Import-SPWeb with this file, but it’s rather tricky to get it to go anywhere other than to overwrite the entire destination notebook. 

Instead, what I prefer to do is:

  1. Rename the .cmp to a .cab
  2. Copy the contents of the .cab out to a folder somewhere.  You should have a few .dat files as well as a Manifest.xml
    image
  3. Open this Manifest.xml and search for the name of the section that you need to recover.  In this node, you should see a property called “File Value”
            <File Url="SiteAssets/OneNote Test Notebook/Section 2.one" Id="14a71719-0dd5-473c-ae95-b07984bb452f" ParentWebId="928b0f47-ae98-4ddb-95d5-10998bb374d3" ParentWebUrl="/OneNoteTest" Name="Section 2.one" ListItemIntId="4" ListId="429659f3-ff6e-4ece-91d7-d4574f98ed1e" ParentId="6ac3f7f9-c609-4e0c-a86e-94abc6aa9295" TimeCreated="2014-08-13T19:57:32" TimeLastModified="2014-08-13T20:02:31" Version="1.0" FileValue="00000001.dat" Author="13" ModifiedBy="13">
  4. Once you’ve found the name of the .dat file, just rename that .dat file to your section name .one (e.g. in the example above “Section 2.one”
  5. Open up that .one in OneNote and do whatever you need to with it (copying specific pages or the entire section back into the original notebook). 

 

Remove User Permissions to Delete

Can be used for: On-premises of Office 365

We have all probably gone through the process before where we create a “Not quite contribute” permission level (basically, a custom permission level that removes the “Delete” permission (as described on TechNet).  In this situation, we’ll create a permission level called “NoDelete”, with the following selected:
- Add Items
- Edit Items
- View Items
- Open Items
You can add additional components there as well, but you’ll want to make sure that you do NOT select any of the “Delete” options.

You’ll want to customize your permissions on the library where your notebooks are stored to remove users’ “Contribute” permissions, and replace them with the “NoDelete” permission level. With this permission level configured, you’ll experience the following:

Action OneNote OneNote Client OneNote Web App
Can create pages Yes Yes
Can create sections Yes Yes
Can rename pages Yes Yes
Can rename section No.  You can change it in the UI, but it will change back at “Sync” Yes
Can delete page Yes (darn…) Yes
Can delete section No. You can change it in the UI, but it will un-delete at “Sync” No.  The UI will let you delete, and the section won’t show up for a while, but if you close and re-open the notebook, it’s still there.

So basically, by removing the “Delete” permission, you can prevent the deletion of a section (which remember, is manifested as a physical file within the OneNote notebook/folder); but pages can still be permanently deleted through the web browser (and sent to the recycle bin via the OneNote client).

 

Make your Notebooks Small and Security Tight

Can be used for: On-premises of Office 365

This isn’t so much a technical solution, but more of a business education one. Many OneNote notebooks will sooner or later grow.  I know at KiZAN, have one Notebook that has accumulated over 500 pages across 12 sections contained within it.  Luckily, this notebook is still on-premises (so we have SQL backups), and we have removed our WOPI Bindings for OneNote editing through the browser, so most of our concerns have been alleviated; but there is still a risk of accidental deletion of entire sections that could take multiple hours to restore. 

Rather than creating one hulking “Company-wide staff meetings” notebook, it would make more sense to create a notebook for each department within an organization, and to control the security to only allow those within those departments to edit the notebook’s contents (but potentially still allowing for a wider audience for read).  The fewer proverbial “cooks in the kitchen”, the better chance that we can get to isolate accidental deletions. 

Think Like a Developer

Can be used for: Office 365 (can be used in on-premises, but only if your SQL backup schedule doesn’t match your anticipated recovery window for OneNote).

I’m going to write up a sample solution for this to be posted in a later post, but long story short, you can basically back up your Notebooks on whatever schedule you see necessary using the SharePoint Client Object Model against a SharePoint Online site.  Basically, you’ll want to store the location of a “critical OneNote Notebook” (and what better place than in a SharePoint list…), and then have an automated job to loop through each notebook (aka folder) and it’s contents, and write them to a local file store.  From here, you’ll then of course want to commit those items to a backup medium and a retention policy, but you will functionally have a frequent backup taken of all notebooks on a regular schedule, that you could use for granular restore of content as you need to do so.

Conclusion

As  you can see above, like most things, one size does not fit all in regards to supporting OneNote within your organization.  Depending on your RPO and RTO, a combination of the above actions may be right for your organization.  This doesn’t mean that we need to be afraid of OneNote, it’s a very powerful tool, and one I will continue to recommend.  Just make sure you understand the risks associated with OneNote, and you plan accordingly. 

Cincinnati SharePoint Users Group - January 2015

Thanks to all who attended my session at the Cincinnati SharePoint User Group tonight!  As promised below is the presentation (including all of the links we looked at) as well as the farm-solution you can use to deploy master pages and related assets to your SharePoint 2013 on-premises environment.

 

Presentation:

Sample Source Code: http://1drv.ms/1BNwUsm

Business Critical OneNote-Part 2: The Sometimes On Recycle Bin?

This post is a continuation of my “Business Critical OneNote” series.  There’s a slightly longer delay between this post and Part 1 in part because I underestimated the time-sink that organizing SPS Louisville 2014 would be, so hopefully part 3 comes much more quickly after this one.   
    Part 1 – Anatomy of a Notebook 
    Part 2 – The Sometimes On Recycle Bin? (this post)
    Part 3 – Practical Recommendations (coming soon)

So, after reading part 1, we now know how a OneNote notebook actually works in SharePoint 2010 and 2013; it’s basically a folder with a bunch of OneNote section files contained within it.  And that’s what can get us into trouble…

The Setup
We have a OneNote notebook on our team site, Project A.  This notebook has two sections, Section 1 and Section 2.  Each section has several pages contained within them. We have two people working on this team site, both with “Contribute” permissions on it; John Delete and Jane Doe.  John and Jane are working on the project together, and keep track of notes for the project within their notebook, using a combination of OneNote in Office Web Apps as well as OneNote 2013 clients. 

We’ll conduct this scenario in both SharePoint Online (as of early November) as well as SharePoint 2013 on-premises 2013 (October 2014 CU) , with identical results; but obviously stuff changes all the time in SPO, so I’ll try to keep this series updated if/when anything changes in either platform than can affect this behavior.

The Results

I’m putting the results before all of the scenarios; since they’re wordy, and probably of little interest unless somebody wants to double check me on my results.  

In summary, deletions made in OneNote Web App (aka OneNote Online) are permanent.  They do not go to a OneNote-specific recycle bin, and they do not go to a site-level recycle bin.  Section deletions disappear entirely, including across sync’ed OneNote clients within a few minutes.  Page deletions disappear, but if you have versioning turned on, you can roll back versions manually (but at a chance of losing other changes made during the editing session). 

Deletions made via OneNote 2013 and 2010 are moved to a OneNote recycle bin inside of the notebook, and can be recovered for 60 days. However, this recycle bin can also be cleared by anybody with contribute permissions to the notebook, and then it’s gone (asides from backups if they exist).  So the OneNote desktop client is safer for users, but still lets contributors potentially delete components from the notebook that you may not want to be deleted. 

The below results have been tested on traditional SharePoint team sites as well as when stored in OneDrive for Business (both on-premises and in SharePoint online), with no major differences other than verbiage or UI.  They have not been tested on consumer OneDrive, I’ll add that to the to-do list.

A quick test of the OneNote mobile apps shows that they behave mostly like the desktop apps, although those change pretty frequently (except for us WP users Sad smile ), so don’t take my word on those ones…

Scenario A – Deleting a Notebook Section in the OneNote Web App

Action On-Premises SharePoint Online
John Delete right-clicks on a section and selects “Delete” A user right-clicks on a section and selects “Delete”
image


OneNote Web App Warns that the deletion is permanent
image



After we click “Yes”, the section is deleted
A user right-clicks on a section and selects “Delete”
image

OneNote Web App Warns that the deletion is permanent
image

After we click “Yes”, the section is deleted
What’s in the OneNote Folder The section has been removed
image
The section has been removed
image
Is it in a OneNote Recycle Bin? There is no recycle bin when in the web client.  If we open it in a desktop application, the recycle bin is empty.
image
There is no recycle bin when in the web client.  If we open it in a desktop application, the recycle bin is empty.
image
Is the section removed from other copies of the Notebook (e.g. sync’ed copies on a desktop) Yes, within about 30 seconds. Yes, within a few seconds
Is it in the Site Recycle Bin? Nope
image
Negative.
image
Can we restore by rolling back a version? No, the section is gone.  Versioning is not applied at the folder (notebook) level, and the delete was final. No, the section is gone.  Versioning is not applied at the folder (notebook) level, and the delete was final. 
Can we restore by an unattended content database attach? You can (at least a recently as the content DB was last backed up). Not applicable in SharePoint Online. 
Conclusion Deleting a OneNote section is permanent (as the warning tells us). Deleting a OneNote section is permanent (as the warning tells us).

Scenario B – Deleting a Notebook Section in OneNote 2013

Action On-Premises SharePoint Online
John Delete right-clicks on a section and selects “Delete” Right click on a section and select “Delete”
image

Prompted with warning “Are you sure you want to move the following section to this notebook’s Recycle Bin”?
image




Right click on a section and select “Delete”
image

Prompted with warning “Are you sure you want to move the following section to this notebook’s Recycle Bin”?
image
What’s in the OneNote Folder Note that the section has been removed from the root-level folder for the notebook

We get a folder, called “OneNote_RecycleBin”. 
image

Inside of here, we see our deleted section, as well as a TOC for the deleted section (and a new section for deleted pages, that is currently empty)
image
Note that the section has been removed from the root-level folder for the notebook

Our OneNote_RecycleBin is there, containing our deleted section, TOC, etc. 
image

Is it in a OneNote Recycle Bin? Yes!  It even syncs across OneNote clients (so both John Delete and Jane Doe see the section in the recycle bin on their clients)
image
Yes, just as on premises, our section is in the recycling bin folder in OnteNote.
Is the section removed from other copies of the Notebook (e.g. sync’ed copies on a desktop) Yes, once it syncs again.  Same as on-premises, it’s all sync’ed up.
Is it in the Site Recycle Bin? No. No.
Can we restore by rolling back a version? The file is gone, but it is in the Recycle Bin (if you get to it through Explorer View) that you can copy/paste as necessary.  Note that there is no ability to restore from the web app or web interface, though.  Explorer view can get us directly to the section in the Recycle Bin for restoration. We don’t have the ability to perform the restore through the OneNote Online web-client, but we can open it up in the desktop client to perform the restore as well.
Can we restore by an unattended content database attach? Not needed, but yes, this is an option if you want to do things a hard way. Not in the cloud we can’t.
Conclusion Deleted sections, when deleted through the OneNote client, will remain in the OneNote notebook for 60 days unless somebody explicitly goes into the OneNote recycle bin and deletes the item from there as well, or empties the recycle bin. SharePoint Online and Office Online perform the same as on-premises in this scenario.

Scenario C – Deleting a Notebook Page in Office Web Apps

Action On-Premises SharePoint Online
John Delete right-clicks on a page and selects “Delete” Right click on a page within a section, and hit “Delete”
image

A warning is displayed “Be careful!  Deleting a page cannot be undone.  Are you sure you want to permanently delete this page?”
image


Right click on a page within a section, and hit “Delete”.
image


We are warned “Be careful! Deleting a page can’t be undone. Are you sure you want to permanently delete this page?”
image
What’s in the OneNote Folder We still have our two section files inside of the folder.  But note that the section with our deleted page shows a modified date that matched our click on the “Yes” button.
image
We still have our section file, with the updated Modified timestamp when the deletion occurred.  However, the page is not in the Recycle Bin (as with on-premises).
image
Is it in a OneNote Recycle Bin? No.  If there was a previous deletion in the desktop OneNote client, the Recycle Bin folder will have a “DeletedPages” .one, but it does not contain the page.
image
Same as on-premises in this case. 
image
Is the page removed from other copies of the Notebook (e.g. sync’ed copies on a desktop) Yes, within 10-30 seconds of the Notebook being opened and synchronized. Same as on-premises.
Is it in the Site Recycle Bin? No. 
image
No.
Can we restore by rolling back a version? No! 
Even if our library has versioning turned on, as below: 
image

OneNote Web App always just overwrites the version as 1.0
image
We get stuck at 1.0 here, as we do on on-premises. 
image
Can we restore by an unattended content database attach? Yes. No.
Conclusion Version history can’t save us here.  A deletion made through the Web App at the page level is permanent. Same as on-premises.  Deletions are permanent.

Scenario D – Deleting a Notebook Page in OneNote 2013

Action On-Premises SharePoint Online
John Delete right-clicks on a page and selects “Delete” Right click on a page in a section and hit “Delete”
image


The page is immediately deleted
Right click on a page in a section and hit “Delete”.  The page is immediately deleted.
What’s in the OneNote Folder The section file has a “Modified” date of when the deletion occurred.  But note that we have our OneNote_RecycleBin folder which now contains a OneNote_DeletedPages section, which contains our page.
image
Same as on-premises.
Is it in a OneNote Recycle Bin? Sure is. And we can restore the file from there within 60 days of the deletion. 
image
Same as on-premises.
Is the section removed from other copies of the Notebook (e.g. sync’ed copies on a desktop) Yes it is, as the Notebooks are synchronized. Same as on-premises. 
Is it in the Site Recycle Bin? No No
Can we restore by rolling back a version? All updates made in OneNote do not append versions. 

Any updates made via Explorer View (e.g. copying the section back and forth from recycle bin to the root level folder) do append versions (as noted below).
image
Same as on-premises.  Updates are pushed to the latest full version.
Can we restore by an unattended content database attach? Yes we can. No (you could have guessed that by now, huh…)
Conclusion Deletions of pages within Notebooks from the OneNote Client can be restored by anybody with a synced desktop copy of the notebook, or content database restored. Deletions of pages within Notebooks from the OneNote client can be restored via the OneNote desktop client (as with on-premises)