SharePoint Retention Policies

Summary

  • If SharePoint Retention Policies don’t seem to be working, first know that they’re based on timer jobs, so run the timer jobs.
  • Retention policies can be set at three different locations. One of the locations overrides the others and it’s easy to miss that fact.
  • PowerShell can be used to set the created/modified dates for testing purposes

Basic Setup

Following explains the basic setup for Retention Policies. If you’re already familiar with this, then scroll down to the timer job section.

Consider the following setup, a site collection that has the following:

  • Custom content type, based on the document type, with an additional DateTime field of “ExpirationDate”
  • Document library set to use that content type

To setup the retention policy, one option is to go to site settings –> site content types –>
your content type
–>Information management policy settings. From here, click on “add a retention stage”. For my demo, I have it configured as:

So, my retention policy states that when the expiration date is reached (which is, again, a custom field I added to the content type), the item will be moved to the recycle bin.

Yes, the above is a brief summary. I you need more, there are quite a few resources available for the basic setup, including:

http://blogs.msdn.com/b/ecm/archive/2011/10/12/policy-and-retention-in-sharepoint-server-2010.aspx
http://office.microsoft.com/en-us/sharepoint-server-help/define-document-retention-policy-HA010095162.aspx
http://weblogs.asp.net/sreejukg/archive/2010/11/11/retention-policy-for-document-library-in-sharepoint-2010.aspx

Timer Jobs

So, if you set up the above, and edit a document to have an expiration date in the past, the document will still remain in the library. The expiration process is actually run via a couple timer jobs. So, the document will remain in the library until both timer jobs run. Specifically, the timer jobs are:

  • Information Management Policy
  • Expiration Policy

Note that the above are set to run once a week by default, so to test, either set the interval to run every few minutes, or just manually kick them off. But, note that the Information Management Policy must run first.

(Most other sources I’ve seen say that both timer jobs default to run weekly. On my box, the Information Management Policy was set to run every 5 minutes, while the Expiration policy was set to run every Saturday night. Also, see the last section of this post for more details on the timer jobs).

Source of Retention

If you’ve already kicked off both timer jobs, but your expired docs are still sitting in the library, be aware that there’s also a confusing setting referred to as “source of retention”. This setting has a huge impact on how the retention policy is processed, but unless you know what you’re looking for, it’s incredibly easy to miss. The result of setting the “source of retention” incorrectly will make it seem like the retention policy is non-functional. What this refers to is the fact that you can set up retention policies on either content types, or on the library/folder itself.

For example:

The lower section of the above screenshot shows that my custom content type named “Policy” has a Retention Policy defined, whereas the Document and Folder content types do not.

However, the top section shows that the “source of retention” is “Library and Folders”. Again, the bottom section is listing the policies for the specified content types. So, the top section is indicating that the bottom section will not be used! To restate this, I have a content type in use in a document library and I have a retention policy set on the content type, but when the timer jobs run, nothing will happen, because the library in question is set to ignore retention policies set on content types, and will only use retention policies set for the particular library! (That’s what the note at the bottom is referring to: “Note: the above listing is not being used”)

So, backing up a bit, here are the possible ways that a retention policy can be set:

Content Type at the Site level

Information management policies can be set for a content type from Site Actions –> Site Settings –> Site Content Types –>
content type –> Information Management Policy Settings. Policies set at this level will apply wherever the content type is used. So, if a policy has already been created, and then a new library is created that uses the content type, that policy will be in effect on the new library, with no further effort required.

Content Type at the Library Level

Information management policies can be set for a content type in an individual library from Library Settings –> Information Management Policy Settings –>
content type. Policies set at this level will apply to any item of the specified content type in the current library. These policy settings will have no impact on other libraries.

Note, it’s difficult to determine which of the above is in use. In the following example, I have a library with two custom content types, each as a Retention Policy defined, one is defined through the Content Type at the site level, the other uses the Content Type at the library level:

Can you tell which is which? No? You can tell by clicking on the links:

In the above, the first link to “Policy” takes you to a page that says you can’t change policy settings here, but to do so you must change the parent, which is a reference to the actual content type. So, “Policy” is being set from the site level, where “Policy2” is being set via a content type policy that is configured at the library level.

Policy at the Library / Folder Level

In addition to being able to specify policies for content types, you can also specify policies for libraries / folders. Policies set at this level override policies set at the content type level. To set this, navigate to Library Settings –> Information Management Policy Settings –>
content type. The top section of this screen might be:

(If you don’t see the above, enable the following site-collection feature: Library and Folder Based Retention.)

From the above, click “Change Source”, and that will take you to the following screen, where you can change the library to ignore content type retention policies, and instead use library policies:

Once you change it to “Library and Folders”, a section will be displayed that will allow you to configure retention policies for this library.

(Note, there’s a bug with the notification. If Folder Based Retention Polices are on, the following message is displayed: “*Note: Since this library is using library and folder retention, all documents will use those schedules. Content type retention policies are ignored.” But, that message will still be displayed if the feature is disabled while a library has Folder Based retention. So in that case, the Folder based retention would not be available, and would not take effect (since the feature was disabled), but the message would still be displayed.)

More Details on the Timer Jobs

Above, I mentioned that both timer jobs would have to be run. That’s not quite true. Following are a few more details I was able to glean from observation:

Information Management Policy

This job analyzes the policies in effect, and creates the calculated field that stores the expiration date.

Expiration Policy

This job processes the policies (moving them to the recycle bin, for example), for any documents where the calculated field is less than the current date.

For example, consider the following:

  1. Policy is created and applied that is based on the Created date of a document.
  2. The Information Management Policy job is run. At this point, there is a new (hidden) calculated field added to the list that contains the date that the policy will take effect. (the expiration date).
  3. The Expiration Policy is run. At this point, any documents that have been expired based on their Created date will be have their expiration policies carried out
  4. The policy is modified to be based on the Modified date.
  5. The Expiration Policy is run. At this point, any documents that have been expired based on their Created date will be have their expiration policies carried out. Again, even though the policy is now for the Modified date, when the expiration policy is run, it will still be for the created date. This will continue until the Information Management Policy job is run and updates the calculated field that determines the expiration date.

Testing Retention Policies

A common method to test retention policies is just to decrease times to a day instead of a year. I know that’s a not a full test, but it’s useful for folks to learn how the system works and what happens when documents expire, and to allow them to go through the full life-cycle of the system. Just remember to set the timer jobs to run each night instead of once a week.

Or, to test existing retention policies, you can use powershell to set the created/modified dates for a document:

$web = get-spweb("http://www.someurl.com/sites/somesite")
$file=$web.getfile("shared documents/somedoc.doc")
$item = $file.item
$item["Created"] = "2005-01-01 10:00:00"
$item["Modified"] = "2005-01-02 11:00:00"
$item.update()

Good luck.

Advertisements

7 thoughts on “SharePoint Retention Policies

  1. Pingback: Information mangement policy settings | SharePoint Q&A

  2. Nazmi

    does retention period only work sequential or can it skip to the next stage when there is no data found in the middle of the stage?

    Reply
  3. Doug P

    Do these apply to on-prem environments, cloud, or both? I know you’re at the mercy of Microsoft for when the timer jobs run if you’re cloud-based so testing is very difficult.

    Reply
    1. Mike G Post author

      This post should mostly apply to both. But as you mentioned, we have no control over the timer job frequency in SharePoint Online. They say they’re run weekly, but that’s all we know.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s