Archiving data in the RDS
Every RDS project allocation contains three spaces:
- live is for data that is being actively worked on, or frequently referenced. The ongoing cost of an RDS allocation is proportional to the amount of data in this space.
- ephemeral for temporary files and data of short-term use. This space is un-metered but all files within it are deleted 30 days after creation.
- archive is for long-term storage of data that will be infrequently-accessed. There is a one-off cost to archiving data in this space.
How to use RDS archiving
When using RDS archiving, it is important to understand the following:
- Files placed in the archive space become permanently read-only 3 days after being placed there. After this time, they cannot be renamed, deleted or altered in any other way.
- Files in archive remain accessible through the same means as live and ephemeral but there will be a delay of up to tens of minutes when accessing an archived file, as the data is brought back from cold storage.
It's therefore important to plan your use of archive and organise data in advance. We recommend first staging the data to be archived in live or ephemeral, organising it under a top level folder. Keep the number of files low - if you have lots of files (particularly if they are ones smaller than ~1MB), it is important to pack these up using Zip or similar tool. The easiest method is to create a single archive file with 7-Zip, which is pre-installed on all College-owned Windows desktop machines. On Linux or macs, the command "zip" can be used. Try to avoid creating Zip files larger than ~100GB.
When you have fully organised the data to be archived, copy it all at once into archive. When the copy has been completed successfully, you can safely delete the staged and original copies of the data.
There is a one-off cost of £100/TB/decade (pro-rated) for all data placed in archive. This is automatically billed to the account code associated with the project allocation in the month following the accession.