Cleanup Workflow History

Hi Tim,

I am test-running this tool and got an error at the very last step when purging site recycle bin
Please see blow log file.

Do you know why is it?

Thanks
Jack





CleanupWorkflowHistory
======================
SiteUrl: http://server1:8086
ListName: Workflow History
NumDays: 60
MaxHistoryListDelete: 5
MaxRecycleBinDelete: 5
PermanentDelete: True
ReportOnly: False
LogFileName: cleanup.log
20/02/2012 2:15:06 AM: started CleanupWorkflowHistory
20/02/2012 2:15:07 AM:  started processing batch.
20/02/2012 2:15:08 AM:   MaxHistoryListDelete: 5
20/02/2012 2:15:08 AM:   5 items deleted
20/02/2012 2:15:08 AM:  finished processing batch
20/02/2012 2:15:08 AM:  started purging Web Recycle Bin
20/02/2012 2:15:08 AM:   5 items in Web Recycle Bin
20/02/2012 2:15:08 AM:   MaxRecycleBinDelete: 5
20/02/2012 2:15:08 AM:   5 items purged
20/02/2012 2:15:08 AM:  finished purging Web Recycle Bin
20/02/2012 2:15:08 AM:  started purging Site Recycle Bin
20/02/2012 2:15:08 AM:   233163 items in Site Recycle Bin
20/02/2012 2:15:13 AM:  ERROR in PurgeSiteRecycleBin: Operation is not valid due to the current state of the object.
20/02/2012 2:15:13 AM:  finished purging Site Recycle Bin
20/02/2012 2:15:13 AM: finished CleanupWorkflowHistory

Hi Tim,
I'd like to know if I can disable the timer job - Workflow Auto Cleanup and only use the command line.
The command line will be delete the instances and related task entries too, or the command line only delete the items of the Workflow History List.
I'm wanting disable the Job and to use only command line with scheduling daily due to necessity keep item for one year (360 days).
Thank you for your help.

I've opened the CleanupWorkflowHistory project file
I set the Target framework to .NET Framework 3.5
I save all
When I choose Build Solution I get errors that the 'SharePoint' namespace does not exist.
How do I clear these up?

Thanks!

Mike, you're probably missing a reference to the SharePoint dll.

Hi Tim,

Do you have any clue on why the script is not deleting items from the 2nd stage recycle bin?

this is what I'm running:

CleanupWorkflowHistory.exe http://site/subsite "Workflow History" 60 5 5 true false test.log

This is the result:

CleanupWorkflowHistory
======================
SiteUrl: http://site/subsite
ListName: Workflow History
NumDays: 60
MaxHistoryListDelete: 5
MaxRecycleBinDelete: 5
PermanentDelete: True
ReportOnly: False
LogFileName: test.log

4/4/2011 12:25:44 PM: started CleanupWorkflowHistory
4/4/2011 12:25:59 PM:  started processing batch.
4/4/2011 12:26:00 PM:   MaxHistoryListDelete: 5
4/4/2011 12:26:00 PM:   5 items deleted
4/4/2011 12:26:00 PM:  finished processing batch
4/4/2011 12:26:00 PM:  started purging Web Recycle Bin
4/4/2011 12:26:00 PM:   5 items in Web Recycle Bin
4/4/2011 12:26:00 PM:   MaxRecycleBinDelete: 5
4/4/2011 12:26:00 PM:   0 items purged
4/4/2011 12:26:00 PM:  finished purging Web Recycle Bin
4/4/2011 12:26:00 PM:  started purging Site Recycle Bin
4/4/2011 12:26:00 PM:   1704296 items in Site Recycle Bin
4/4/2011 12:47:23 PM:   MaxRecycleBinDelete: 5
4/4/2011 12:47:23 PM:   0 items purged
4/4/2011 12:47:23 PM:  finished purging Site Recycle Bin
4/4/2011 12:47:24 PM: finished CleanupWorkflowHistory

I'm tryng to figure out why items are not being deleted from the recycle bin :(

I also came across the same scenario and learned that when we use SPRecycleBinItem.Delete() on first stage recycle bin, item gets permanently deleted. It wont be going to the second stage recycle bin and there for does not require a cleanup of 2nd stage recycle bin.

Also one more point I would like to add is that we have to specify which recycle bin to query by setting  

query.ItemState = SPRecycleBinItemState.FirstStageRecycleBin;

otherwise we cannot make sure that we querying the correct recycle bin.


Please correct me if I am wrong

Thanks

Charls & Luis - I haven't ran into a situation where workflow history items weren't deleting from the 2nd stage recycle bin.  This tool will only delete items in recycle bins that are from the Workflow History list you provide in the parameter - that is what this line determines:
if (item.DirName.Contains(_ListName)) // item is from the Workflow History list 

To be honest, I don't remember the reason why I check both recycle bins.  I'm sure there was a reason, but it has been a while since I worked on this project.

Thanks,
Tim

Trying to understand why it is so important to cleanup workflow history?  We have a site that has 569,403 records.  Should we use your tool?

You have to decide whether you want to clean up the Workflow History list.  In my situation, it was impacting backup operations & workflows were failing intermittently on new InfoPath forms that were submitted to the library.  After I cleaned up the Workflow History list, it resolved both of these issues. 

Hi Tim, does the SharePoint search service have to be enabled to run this program?

We have the SharePoint search service disabled and when I try to run the tool - the first thing I get is "Unable to connect to site: URL to subsite - value does not fall within the expected range.

Hi ISharePoint,

No, you don't need to have the SharePoint search service enabled to use this tool.  Double check the URL 

Hi ISharePoint,

No, you don't need to have the SharePoint search service enabled to use this tool. Double check the URL and the name of the list you use when calling the tool.  That error indicates that one of them is incorrect.  For example:

CleanupWorkflowHistory.exe http://moss/site "Workflow History" 5 5 5 false true MossTest.log

Thanks,
Tim

Thanks Tim, I just realized I misread the syntax, what should be in the double quotes is the display name of the list, not the listname from the URL (ours is different for whatever reason).

One more question, does the workflow history list or any list so say to speak store data in both the AllUserData and the ALLDocs table in the content db? I just wanted to make sure by purging this list I take care of the many millions of rows of data in both tables. Thanks a bunch!

Hi ISharePoint,

I'm not sure where list items are stored in a content database.  You could do some testing to figure that out, but the purpose of this tool is to manage the size of Workflow History lists.

Thanks,
Tim

Hello,

We are tryng to use your code but no luck, I see a couple of .exe files under bin and obj but neither of them seem to work.. is there something else we need to do in order to make your code to work?

thanks!

Hi Luis,

Copy CleanupWorkflowHistory.exe to your SharePoint server.  It is a console application, so you will have to run it from the command line and use info specific to your environment.  For example:
CleanupWorkflowHistory.exe http://moss/site "Workflow History" 60 100 100 false true Cleanup.log

The values for the switches are explained in the Usage section on the Description page.

Thanks,
Tim

The executable did not work for me also on SharePoint 2010  but I found a very helpful Microsoft tech who helped me  rebuilt the program using Target Framework: .NET Framework 3.5 within Visual Studion 2010.

It worked fine SharePoint 2010 after the rebiuld on VS 2010. I ran a small test to delete 10 items and worked fine. Now I plan to change the content database to Simple Mode and run it as a Windows scheduled task to delete 10,000 items every 15 minutes over the weekend. I have 700,000 to delete.

rcusa - thanks for the info.  

SP2010 targets the .NET 3.0 framework, so it makes sense you would have to rebuild the project using .NET 3.0 or 3.5.  SP2007 targets the .NET 2.0 framework, which is what I used in the VS2005 solution I uploaded.  Glad you found it useful - keep an eye on the size of your SQL transaction logs.

Thanks,
Tim

Hi Tim,
Yes, I used the CleanWorkflowHistory program to clear down over 23 million items during the Christmas 2010 holiday.

Initially I rebuilt the program using Target Framework:  .NET Framework 3.5 within  Visual Studion 2010 Premium.
Then ran a number of read-only runs against a test web site to see how it works; deleted some items from the test site.
On the live site, initially ran it in read-only (Reporting mode); then ran it to delete a small number of items on the site.

This worked, so I ran it every 6 minutes to delete batches of 4,000 items, at one stage deleting 10,000 items every 8 minutes.
The legacy Workflow History list contained over 23 million items!  (custom outsourced workflow with verbose logging).
Fortunately, we were not running the workflow site on our new SharePoint 2010 server, so we could delete all items over 10-12 days!  (The items were included in a restore of our SharePoint 2007 server that currently runs the workflow.)

The transaction log grew to a large size overnight (about 50GB) and so needed some management.   Backed this up; set model to Simple, shrunk the log, set model back to Full...
So your program proved invaluable.

Many thanks,
Brian Sexton

Thanks Brian - glad you found it helpful!