Nobody re-reads a research dataset for the reasons it was created.

Facilities converge on three tiers, and most complaints about research storage are really data sitting in the wrong one.

Scratch is not a small archive. It is a fast, expensive job workspace with a purge schedule attached, and letting it quietly become permanent ruins a facility's economics.

An illustration, with invented figures. Substitute your own instrument or job logs before deciding anything.

Take a beamline or simulation campaign producing 4 TB per run, executed 250 times a year.

• Annual production: 4 × 250 = 1,000 TB (1 PB) per year
• Suppose 20% is reduced, described, citable output: 1,000 × 0.20 = 200 TB to archive per year
• The remaining 80%: 1,000 × 0.80 = 800 TB regenerable from inputs and code, purgeable from scratch
• Over a five-year cycle: 1 PB archived, against 4 PB that never had to leave scratch

That ratio is the budget conversation. Archive all of it and the archive is five times larger for no research gain. Archive none of it and 1 PB of irreplaceable output waits on the costliest storage on site until something deletes it. The percentage varies sharply by discipline, so derive it per group.

Scratch fills, the queue backs up, a purge is announced, deferred, softened, then enforced badly. It repeats because the two sides carry different risks: the facility pays for the capacity, the research group loses the work. Policy statements alone have never fixed this. Making the exit path easier than doing nothing does.

• Automate movement. If getting data off scratch depends on somebody remembering to copy it, it will not happen.
• Leave the path where it was. A migrated file still visible at its original location does not break last year's job scripts.
• Publish a clock and keep it. A purge date that never slips does less damage than one that slips unpredictably.
• Separate regenerable from irreplaceable. Checkpoints can go. Detector output from a sample that no longer exists cannot.

Ownership of the purge policy stays with the institution. The storage layer's contribution is making the compliant choice the low-effort one.

A parallel filesystem is tuned for concurrent throughput, not cold data, and its metadata services degrade first when it fills with files nobody has touched in two years. The design question is how data exits without a gateway becoming the choke point.

QStar announced an integration with the BeeGFS parallel filesystem in June 2026. Per that announcement, QStar Network Migrator, a hierarchical storage management product, uses the BeeGFS Data Management API to read file metadata directly instead of walking the tree, which the vendor states reduces scan time and impact on primary storage. Policies can be written against last access time, ownership, group membership, size or file type. Files move while access continues through lightweight links or stubs.

Underneath, QStar Archive Manager supplies NFS archive gateways with caching, targeting tape libraries, object storage and cloud. For shared national or multi-faculty facilities, QStar's Global ArchiveSpace datasheet describes a clustered active archive with a global namespace, front-end access over SMB, NFS, S3, HTTP and FTP, and a Physical Library Manager that pools tape drives across nodes rather than partitioning the library. One inconsistency in that document: the highlights state 3 to 64 nodes while a diagram shows 2 to 64. Confirm the floor before sizing. See tape and LTO systems for the media layer underneath.

A citation is a promise that an identifier still resolves years after the storage under it was replaced. Routine infrastructure work breaks that promise quietly. Three things hold it together.

1. An identifier that does not encode a location. If the cited handle contains a hostname, mount point or bucket, the first migration invalidates the paper.
2. A namespace stable across media generations. Archive media has a shorter service life than the citation. Virtualising the archive behind a filesystem or S3 interface lets content move underneath while the presented path does not.
3. Fixity evidence. A checksum captured at deposit and re-verified on a schedule shows the bytes are the cited ones, not just that a file of that name exists.

Funders and institutions commonly attach availability conditions to grants, and those conditions outlast the hardware. Nothing here states what any funder, regulator or policy requires of you. That reading belongs to your research office and legal advisers. Our concern is that a resolution request in year twelve returns the right object.

Model reproducibility is stricter than dataset citation because the answer must be exact. "Trained on the genomics corpus" is not an answer. "Trained on this frozen snapshot, manifest hash included, on this date" is.

• Freeze the training set. A corpus still receiving files is a directory, not a training set. Snapshot it, write it once, reference the snapshot from the run.
• Store the manifest with the model. Keep the file list and its checksums beside the model artefacts, so membership can be confirmed without trusting a path.
• Retain superseded versions. A question about a decision made under version 3 arrives after version 7 ships, which is what cheap deep storage is for. Long-term data archiving covers that tier.

Write-once snapshots are designed to prevent a corpus drifting under a model. They say nothing about whether the model is accurate, fair or safe, and no storage design can.

Research capability across the region is being built quickly, and compute procurement moves faster than data policy. A cluster is specified, funded and installed while the archive is still a future phase. The gap surfaces around eighteen months in, when scratch is full and the first finished projects have nowhere to go.

Two consequences follow. The archive is then sized against data that already exists rather than a growth model, so it arrives undersized. And files deposited during the gap carry no consistent metadata, which is expensive to retrofit.

Adjacent problems appear in oil and gas, where decades-old survey data must stay re-ingestible, and in media and surveillance, where the archive is sized by ingest rate. Browse the other industries we work in, or request an assessment with your annual production figures and current scratch policy.