SpikeForge 1.1.0rc2

1

Executive summary

The original SpikeForge implementation contained structural, allele-accounting, molecule-selection and verification defects that made its output unsuitable as validation evidence. The corrected implementation was evaluated with legacy-versus-corrected BAM comparisons, a six-variant VAF ladder, record-level reconciliation, independent BAM parsing and indexed fetch, and release acceptance tests.

All 34 inspected legacy BAMs failed at the first declared BGZF block boundary. All 20 corrected BAMs passed a boundary walk across 284,860 declared blocks, and all 20 associated BAI files passed structural traversal. Across 114 matched successful variant comparisons, mean absolute requested-VAF error decreased from 0.000915 to 0.000663, a descriptive reduction of 27.6%. The corrected implementation also refused to publish controls when the requested dose could not be physically represented.

2

Study question and hypotheses

Primary question

Can corrected SpikeForge generate structurally valid, correctly dosed and fully auditable BAM controls while refusing infeasible experiments instead of issuing misleading PASS results?

Working hypotheses

1. Structural validity: corrected BAM and BAI files will pass structural checks that legacy outputs fail.
2. Dose accuracy: corrected outputs will have lower absolute requested-versus-achieved VAF error on matched successful events.
3. Auditability: every output difference will be attributable to an authorized edit or declared synthetic record, with no unexpected additions or removals.
4. Fail-closed behavior: infeasible controls will be withheld before publication rather than represented as successful.
5. Fusion realism: planted BAM-level fusion records will be sufficiently similar to nearby real alignments for blind validation use.

These hypotheses summarize the post-audit validation objectives; the study was not preregistered. Hypothesis 5 was not supported.

3

Methods

Study design

A frozen copy of the legacy implementation was compared with the corrected working implementation. Real-data comparisons used fixed inputs, a fixed six-event hg19 ladder and random seed 2026. Structural validation, allele-dose measurement, feasibility behavior, exported-artifact reconciliation and software acceptance were evaluated separately.

Experimental procedure

1. Audit legacy BAM framing, reference validation, allele accounting, molecule identity, tag maintenance, verification, publication transactions and truth artifacts.
2. Apply corrections while keeping the legacy source frozen for comparison.
3. Run the fixed VAF ladder on the historical and stress cohorts. Measure baseline and achieved ALT support directly.
4. Walk declared BGZF boundaries and BAI structures independently of the writer.
5. Reconcile input and output BAM records against exported ledgers and raw-record SHA-256 digests.
6. Run exact-count and remote-input acceptance tests, including independent Bamnostic parsing and indexed fetch.
7. Publish output only after the private candidate passes the verifier and allele-count gate.

Primary and secondary endpoints

5

Conclusions

1. The legacy implementation should not be used as validation evidence; all 34 inspected legacy BAMs had invalid BGZF framing.
2. The corrected variant path produced structurally valid files in the reported checks, improved requested-VAF accuracy on the matched successful subset and correctly rejected infeasible controls.
3. Record-level ledgers and digests accounted for all differences in the representative audit, with no unexpected additions, removals or modifications.
4. Exact-count mutation insertion is supported by focused tests and independent parser acceptance when molecular-family constraints permit the requested count.
5. The fusion-realism hypothesis was not supported. BAM-level no-aligner fusion outputs should not be used as blind-realistic controls.

6

Limitations

• The VAF comparison includes only events that completed successfully in both implementations; observations within a library are correlated.
• Native samtools, Picard, IGV and the intended production variant caller were not run in the final Windows acceptance environment.
• Bamnostic provides independent parsing and index use, but does not replace validation with the production HTSlib/Picard toolchain.
• BAM editing evaluates the aligned-read stage. It does not test extraction, library preparation, sequencing, trimming or alignment.
• The aggregated cohort results are not analytical sensitivity, specificity, limit of detection, clinical sensitivity or clinical utility.
• Validation BAMs are intentionally excluded from the public release to avoid distributing operational or potentially identifying sequencing data.
• The release was accepted on Windows. The Python design is portable, but equivalent macOS and Linux acceptance runs remain outstanding.

7

Future directions

1. Production-stack interoperability: run representative outputs through samtools quickcheck, samtools flagstat, HTSlib re-indexing, Picard ValidateSamFile, IGV and the intended caller.
2. Blinded recovery study: lock truth before analysis, spike replicated VAF ladders across depths and library types, and measure caller recovery with confidence intervals.
3. Full-pipeline controls: create FASTQ-level controls and pass them through trimming, alignment, duplicate handling and calling.
4. Fusion development: use the production aligner to generate truthful alignment scores and validate multiple fusion architectures.
5. Molecular feasibility: quantify exact-count feasibility by depth, UMI structure, duplicate-family size and baseline ALT burden.
6. Cross-platform reproducibility: add locked Windows, Linux and macOS acceptance manifests with native-tool outputs.