smartData Protection & Archiving

Public sector records

In government archiving the hard question is rarely how much, it is who is allowed.

Aban Smart designs archive infrastructure for public bodies where data location, classification handling and disposal authority constrain the design before capacity does.

We build to the retention schedule and classification scheme your records authority issues. We do not interpret what any statute or regulator requires of you.

A ministry archive and a broadcaster archive can hold the same number of terabytes and require entirely different infrastructure. The broadcaster is solving for throughput. The public body is solving for permission: which staff may see which classification, whose signature releases a record for destruction, and whether the bytes may sit on equipment outside a defined jurisdiction. Get those three answers wrong and no amount of capacity planning rescues the design. Get them right and the sizing exercise that follows is comparatively ordinary.

Industry data flow, ingest to recallData sources Case management, Document systems, Email & records ingest at Mixed classification. A retention driver, Retention schedule and disposal authority, sets the policy that places data across Active records, Retained records, Permanent transfer. The expected recall outcome is Retrieval on request, within service targets. Tiers shown in green are governed, protected states.SOURCESINGEST & POLICYTIERSRECALLCasemanagementDocumentsystemsEmail &recordsMixedclassificationDRIVERRetention scheduleand disposalauthorityActive recordsOnlineRetained recordsWORMPermanent transferNational archiveOUTCOMERetrieval onrequest, withinservice targets
Conceptual industry data flow: Case management, Document systems, Email & records ingest at Mixed classification, retained under Retention schedule and disposal authority, placed across Active records, Retained records, Permanent transfer, with the expected outcome Retrieval on request, within service targets. Figures shown are illustrative examples, not verified customer measurements. Final topology depends on verified product compatibility.

Sovereignty and data location

Public-sector procurement documents increasingly specify where data may physically reside, who may hold the encryption keys, and which personnel may perform administrative operations on the platform. These are architectural constraints, not preferences, and they eliminate large parts of the commercial storage market before evaluation begins.

Three questions decide most of the shortlist:

  • Where does the data rest? On-premises in a facility the body controls, in a national or government cloud region, or somewhere a supplier will not commit to in writing. The third answer usually ends the conversation.
  • Where does the metadata rest? Catalogue, index and telemetry often leave the country even when the data does not. Management planes that phone home are a frequent late discovery.
  • Who can operate it? Remote vendor support that requires screen access to a classified system is a control problem regardless of contract wording.

Offline media has a property that appeals here: a written optical platter or a tape cartridge in a vault has an unambiguous physical location and a custody record. There is no ambiguity about which jurisdiction holds it. That is one reason optical archive systems still appear in public-sector designs long after commercial buyers moved on. It is a trade against retrieval latency, and worth making only where the classification justifies it.

Records classification and disposal authority

Most government estates are mixed. The same department holds openly publishable material, internal working documents, personal data on citizens, and a small volume of restricted material, frequently in the same file share. Archiving that estate as one undifferentiated pool forces the whole thing to the highest control level, which is expensive, or to the lowest, which is unacceptable.

Classification therefore has to survive ingest. If the archive cannot carry a classification label as durable metadata alongside the object, the label lives only in the source application and disappears the moment the record is migrated. That failure is discovered years later, usually during a transfer exercise.

Disposal authority is the second half of the same problem. In commercial organisations an expired record can often be deleted by the system owner. In public bodies, disposal of a record is typically an act requiring named authorisation, sometimes from outside the department holding it. The archive has to reflect that. Expiry raises a request; it does not execute a deletion. Where a body has no standing disposal authority, the practical consequence is that nothing ever expires and the archive grows monotonically, so the sizing model should assume that until told otherwise.

Sizing a mixed-retention estate: a worked illustration

The following is an illustration constructed to show the arithmetic, not a measurement from any deployment. Assume a records estate of 240 TB currently held on primary storage, growing 18 TB a year, and a schedule that sorts it into three classes.

Class (illustrative) Share of estate Volume today Behaviour after period
Records of enduring value, permanent 15% 36 TB Never expires; transfer candidate
Long-term administrative, 30-year 25% 60 TB Expires, subject to authorisation
Operational, 7-year 60% 144 TB Expires, subject to authorisation

Now project ten years, holding the same 15/25/60 split on new material.

  • New data over ten years: 18 TB × 10 = 180 TB. Total ingested: 240 + 180 = 420 TB.
  • Permanent share of that total: 420 × 0.15 = 63 TB. None of it can ever leave.
  • Operational 7-year material written in years one to three becomes eligible for disposal within the window: roughly 18 × 3 × 0.60 = 32 TB. The 240 TB already on the books is excluded here, because its age profile is unknown.
  • If disposal authority is exercised promptly, the estate at year ten is about 388 TB. If it is not, it is 420 TB.

The 32 TB gap is small. The point of the calculation is not the number, it is that 63 TB of the estate is permanent by definition and will still be there when the current hardware, the current file formats and the current staff have all been replaced. That subset justifies a different tier, a different media strategy and a documented refresh cycle. The 7-year material does not. Sizing one tier for the whole 420 TB overprovisions the durable layer by nearly seven times.

Records of enduring value and transfer out

Permanent retention is not the same as long retention. A 30-year record has an end date and a hardware generation or two to survive. A permanent record has neither, and the design assumption must be that it will be moved between platforms repeatedly.

That argues for a small set of choices made early: open, documented container formats rather than application-native ones; a catalogue that describes the record independently of the storage holding it; and fixity information stored with the object so that a copy verified in 2040 can be shown to match what was written today. The approach is covered in more depth on long-term data archiving.

Transfer to a national archive adds a further requirement. The receiving institution will specify its own package structure, its own metadata profile and its own integrity manifest. An archive that can export a self-describing package with checksums makes that transfer a project. One that cannot makes it an excavation.

Procurement, tenders and the multi-year horizon

Public tenders run on timescales that shape technical design. A specification written in one budget year may be awarded in the next and delivered in the one after, by which time drive capacities and media generations have moved. Two habits help.

First, specify capability and interface rather than part numbers where the requirement permits it. A tender that names a specific drive model can become unbuildable before award. Second, expect to demonstrate the retention behaviour rather than assert it. Public-sector evaluation increasingly includes a witnessed test: attempt a deletion inside the retention period, show the refusal, show the log entry. Design for that test from the start rather than discovering it at acceptance.

Aban Smart works with QStar archive software, INCOM StorEasy WORM appliances and DISC optical libraries. We design, supply, integrate and support these systems. We are not the authors of your retention schedule, and nothing we install makes a body compliant with any named regulation on its own. Compliance depends on your classification scheme, your process and your legal advice.

Related estates with comparable constraints include healthcare imaging, where retention is long and clinical, and energy and upstream data, where asset lifecycles outlast platforms. See the industries overview for the full set, or request an assessment with your current classification breakdown.

Frequently asked questions

We can design so that data at rest, its metadata and its management plane all sit within a boundary you define, and we can document where each component runs. What we cannot do is guarantee the behaviour of software we did not write. The honest approach is to enumerate every component that makes an outbound connection and let your security team accept or reject each one.

Turn your requirement into a defensible architecture

Share the workload, capacity, retention, access, and resilience requirements. Aban Smart will identify the next discovery inputs and the appropriate engagement path.