#0022026-03-19
Indexing Arc from Genesis
What I builtA backfill from block 0 to head, and a live tail that follows new blocks with a two-block lag.
Why it mattersCoverage from genesis is what makes the datasets trustworthy for research.
ArchitectureTwo workers: a range backfiller and a head follower sharing one write path.
ImplementationRate-limited RPC with adaptive batch sizing; checkpoint every 1,000 blocks.
Problems encounteredRPC providers return inconsistent trace formats between versions.
What I learnedPin the node version and record it in the lineage table.
What's nextDesign the raw layer schema (#003).