[Dspace-tech] cleanup assetstore

[Jose Blanco]

It seems like my cleanup routine is not working. As soon as it tries to
delete the 1st item it needs to delete it gets the following error:



DEBUG org.dspace.storage.bitstore.BitstreamStorageManager @ Deleted
bitstream 47749 (file
/l1/dspace/repository/dev/assetstore/59/82/63/598263743033612961148868350549
64859950) with result false

2006-09-21 16:24:14,989 FATAL org.dspace.storage.bitstore.Cleanup @ Caught
exception:

java.lang.NullPointerException

at
org.dspace.storage.bitstore.BitstreamStorageManager.deleteParents(BitstreamS
torageManager.java:740)

at
org.dspace.storage.bitstore.BitstreamStorageManager.cleanup(BitstreamStorage
Manager.java:673)

at org.dspace.storage.bitstore.Cleanup.main(Cleanup.java:67)



4886835054964859950dspace/repository/dev/assetstore/59/82/63/598263743033612
9611

ls:
/l1/dspace/repository/dev/assetstore/59/82/63/598263743033612961148868350549
64859950: No such file or directory



It seems like the bitstream it's trying to delete is not there and this
causes the program to terminate. I looked in the archive for a solution to
this problem, but was not clear what to do. Is there a way to get cleanup
to work?



Thanks!

[Jeroen Ruigrok van der Werven]

Hi Jose,

Post by Jose Blanco
It seems like my cleanup routine is not working. As soon as it tries to

Which version of DSpace is this?

In the past few weeks I've become rather experienced in solving assetstore
cleanup problems since I had over 1 million files in my assetstore and cleanup
also constantly bombed out on me. Now after some fixes to the source code
(which are in the project source tree) it works as it should.

[Jose Blanco]

Jeroen,

I'm running versi0n 1.3, and I'm working on getting 1.4 up and running, and
one of the things I need to do to move from 1.3 to 1.4 is backup the asset
store, and I'm trying to clean it up before I back it up, because it's too
big right now, and really is not suppose to be that big. The 1.3 cleanup
routine has just not been doing its job.

I did see that in 1.4 you the cleanup routing has this bit of logic that may
just solve my problem:

// if the file was deleted then
// try deleting the parents
// Otherwise the cleanup script is set to
// leave the db records then the file
// and directories have already been deleted
// if this is turned off then it still looks like the
// file exists
if( success )
{
deleteParents(file);
}

In 1.3 it calls deleParents even if success is false. So should I just add
this to my 1.3 code for the time being till I get 1.4 working? I just want
to make sure if I add this bit of code, I won't break anything.

Many thanks for taking time to help me with this!
Jose

-----Original Message-----
From: Jeroen Ruigrok van der Werven [mailto:***@ubib.eur.nl]

Sent: Friday, September 22, 2006 3:31 AM
To: Jose Blanco
Cc: dspace-***@lists.sourceforge.net
Subject: Re: [Dspace-tech] cleanup assetstore

Hi Jose,

Post by Jose Blanco
It seems like my cleanup routine is not working. As soon as it tries to

Which version of DSpace is this?

In the past few weeks I've become rather experienced in solving assetstore
cleanup problems since I had over 1 million files in my assetstore and
cleanup
also constantly bombed out on me. Now after some fixes to the source code
(which are in the project source tree) it works as it should.

[Jeroen Ruigrok van der Werven]

Hi Jose,

Post by Jose Blanco
I'm running versi0n 1.3, and I'm working on getting 1.4 up and running, and
one of the things I need to do to move from 1.3 to 1.4 is backup the asset
store, and I'm trying to clean it up before I back it up, because it's too
big right now, and really is not suppose to be that big. The 1.3 cleanup
routine has just not been doing its job.

In my case it was a 1.2.1 installation.

Post by Jose Blanco
I did see that in 1.4 you the cleanup routing has this bit of logic that may
// if the file was deleted then
// try deleting the parents
// Otherwise the cleanup script is set to
// leave the db records then the file
// and directories have already been deleted
// if this is turned off then it still looks like the
// file exists
if( success )
{
deleteParents(file);
}
In 1.3 it calls deleParents even if success is false. So should I just add
this to my 1.3 code for the time being till I get 1.4 working? I just want
to make sure if I add this bit of code, I won't break anything.

I doubt that is the right fix though. You problem seems very much connected to
what I had.

(NOTE: all these have been committed to the source tree and will thus be
present in the next 1.4.x release or whatever the number will be.)

Try this patch
https://sourceforge.net/tracker/?func=detail&atid=319984&aid=1549758&group_id=19984
Perhaps for 1.3 it might need a smallish tweak.

You might also want to add
https://sourceforge.net/tracker/?func=detail&atid=319984&aid=1549867&group_id=19984
and
https://sourceforge.net/tracker/?func=detail&atid=319984&aid=1551876&group_id=19984

These solved my cleanup problems and brought my 1 million+ files assetstore
back down to ~20000 files. And the item table from 600000+ rows to ~16000.

Hope this helps!