Written by Daniel Hickmore, Health Subject Matter Expert, Arkivum
Traditional methods for managing the growing deluge of health data simply do not scale any more. Whether observational, research data or medical, clinical imaging, telemetry data or patient-generated consumer device data, healthcare organisations face increasing risks, compliance obligations and data-related IT management tasks.
As a result of the variety and scale of data there are many long-term data management challenges that the NHS faces as a big data holder. More of its processes are digital, therefore it becomes critical to build in effective governance over access to and sharing of data, compliance with regulations and long-term bit-level data integrity in order to identify, quantify and mitigate any associated risks.
Any systemic deficits in the management of the longitudinal patient data lifecycle can be exposed by new regulations, the need for quality reporting and future patient engagements. Given the value of healthcare data, new approaches to Information Lifecycle Management (ILM) are essential. Whilst scarce resources typically limit healthcare organisations’ ability to handle this internally, the productivity benefits which drive integrated IT services can also create unacceptable operational risks without specialist products and services.
First, it is essential to understand and adhere to your ILM process. For those not familiar, ILM is a comprehensive approach to managing the flow of an information system’s data from creation and initial storage to the time when it becomes obsolete. Unlike earlier approaches to data storage management, ILM involves all aspects of dealing with data, starting with user practices, rather than simply automating storage procedures. By comparison to older storage-based methodologies, ILM should develop a chain of custody and more complex criteria for storage management which both help to understand the data going into the archive, and to provide effective governance over it.
NHS Trusts are responsible under the Public Records Acts, the Data Protection Act 1998, and the Freedom of Information Act 2000 to ensure that all records, manual or electronic, personal or non-personal are created, maintained, used and disposed of in line with the requirements of these Acts. ILM policy forms part of the requirements of the NHS Information Governance Toolkit and is an important component in any healthcare organisations’ data management strategy.
The ILM policy sets out standards for meeting these needs. It provides best practice guidelines for recordkeeping, including all health and non-health information captured in any document format or media type and file size, as well as establishing the security and confidentiality of the Trust’s data. ILM solutions automate the processes involved, typically organising data into separate tiers according to specified policies. The ILM approach recognises that the importance of any data does not rely solely on its age or how often it’s accessed. Users can specify different policies for data that declines in value at different rates or that retains its value throughout its life span.
Trusts understand the importance of the data lifecycle, and due to the various risks accrued over a long-term storage retention period, best practice should involve a risk assessment of their particular organisation’s data lifecycle. Long-term preservation and management of digital datasets is a specialist process. Arkivum has worked within the healthcare sector for many years, leveraging a combination of technology and specialist expertise to deliver risk mitigation via long-term data preservation solutions. We have worked on both clinical and research-based data projects and we are compliant with the NHS IG Toolkit, delivering assurance with our 100% data integrity guarantee and third party escrow. In particular we specialise in working with healthcare institutions who generate high amounts of data and need to preserve this data for many years to come.
As an example, Bristol Genetics Laboratory (BGL), part of Severn Pathology Services, generates digital imaging and gene-sequencing data from over 30,000 genetic investigations every year using molecular and cytogenetic techniques to analyse a wide range of sample types. These investigations result in vast data sets, exacerbated by the fact that as testing gets cheaper, clinicians are carrying out more investigations in order to diagnose more patient conditions successfully. We helped BGL implement a centrally managed, long-term data archiving service that has enabled it to:
- Secure the long-term value of genomics data
- Identify and mitigate long-term data preservation risks
- Reduce backup costs by removing large volumes of data from the backup window
- Reduce the load on other stretched NHS IT departments who have their own cost and efficiency requirements
Today the system is fully compliant with the stringent requirements of the NHS Information Governance Toolkit and is delivered via the North Bristol NHS Trust N3 data network portal to assure the highest standards of data security. We have ensured that BGL can scale to larger storage volumes rapidly and without the need for any further capital outlay. BGL is also seeing significant efficiency savings from the automated ingestion of data from local infrastructure to the Arkivum service. These automated processes replace manual ones that were carried out by a range of different departments, each of which have unique requirements that previously had to be carried out by a specially trained individual.