Curation process

All acquisitions to the Archive undergo a number of steps to prepare them for sharing and re-use. We always contact data owners or creators for advice and clarification if queries arise during processing.

Step 1 – Transfer of data

Before data arrive at the Archive, staff liaise with government departments, researchers and other data owners/creators to make sure data arrive at the Archive in the best possible shape for processing. This is known as custody transfer.

Step 2 – Assigning processing standard

We assign one of four processing standards (levels) to each data collection as they arrive, which determines the work we will undertake. The standard depends on:

• the nature and condition of the data
• the quantity and complexity of the documentation
• the estimated level of use
• whether or not the data collection is to be made available through online browsing tools

Step 3 – Data processing

Before processing, we examine the data against the documentation supplied and, where appropriate, we QA data and metadata – for example, checking labels and the overall integrity of the data. We carry out privacy and confidentiality checks, and recommend an anonymisation strategy, in collaboration with the depositor. We prepare code books for survey data and data listings for qualitative and historical studies, along with archival and dissemination versions of the data. We also carry out enhanced labelling, grouping of survey variables and XML mark-up for interview texts for the highest processing levels for publishing in our online systems.

Step 4 – Documentation processing

Documentation is the material users need to make the best use of the data. If documentation supplied by the data depositor is incomplete, we request more, or where possible, create it ourselves. We prepare the materials into a relevant, usable format, such as a collated online user guide. We prepare more detailed documentation for enhanced user resources.

Step 5 – Metadata creation

We carry out metadata creation at the same time as steps 3 and 4, and this often continues after they are complete. We create a metadata record based on information provided by depositors and on additional work by our specialist cataloguers. This catalogue record is based on the Data Documentation Initiative (DDI) and allows our collections to be searched and for our citation record to be created.

Step 6 – Additional user information

Once the main work has been completed on preparing the data and documentation, some additional documentation is prepared, including a 'Read' file which gives details of the processing levels of the data collections. Depositors are given the opportunity to see what we will be archiving and making accessible.

Step 7 – Publishing data

Once data and documentation are prepared, the material is transferred to our preservation system. Files added are verified and integrity checked. Zip bundles are prepared for users. The catalogue record is then published and the data become available to users. A DataCite DOI is created for every collection, taking the form, e.g. 10.5255/UKDA-SN-3314-1. Versioning is used to indicate new releases of the data.

Step 8 – Delivering data

Most of our data is delivered via a download system, linked to our resource discovery system. We administer data access and manage the conditions associated with access of all data collections. We also deal with most user enquiries ourselves, and keep records of the use of data for administrative and strategic purposes.

Step 9 – Preserving data

We manage the preservation and curation of data using best practice. We keep multiple back-ups of data, in multiple versions, in secure locations to ensure no data loss. We follow a strategy of migrating data and other files when new formats become commonplace. Since 2010, our organisation has been ISO 27001 certified for information security. A number of government departments have also carried out surveillance visits to the Archive to clarify our information security regime.