Announcing Deep Index Dive

7 min readJul 22, 2025

Exploring the Hidden Depths of Your Blockchain Data

Published: July 22, 2025
Tags: [trueblocks], [tools], [blockchain-archaeology]

Have you ever wondered what lies beyond the visible surface of your blockchain index? You know, in those vast digital depths where data drifts like forgotten dreams through ethereal currents? There, beneath endless layers of accumulated blocks, only an index holds both luminous treasures and shadowed mysteries — a realm where beauty and terror dance together in the sublime darkness.

Well, gather your courage and let me tell you about an expedition tool we’ve been crafting that we’re calling Deep Index Dive (formerly known by the decidedly earthbound name chunkDiagnostics).

The Blockchain Archaeologist’s Deep Sea Expedition

If you know us at TrueBlocks, you know we’re obsessed with data integrity. Not in a surface-skimming way, but in the way that deep-sea explorers become obsessed with mapping every trench, cataloging every artifact, and ensuring no ancient treasure gets lost to the abyss. We’ve been diving into local blockchain indexes for years now, and one thing we’ve learned is this: data has a way of sinking into corruption when you’re not watching the depths.

Your manifest claims a chunk contains ancient artifacts worth 16MB of precious data, but when you dive down to investigate, you find only empty treasure chests. Your index charts say there are ruins spanning block range 1000–2000, but your sonar readings (bloom filters) think they’ve found structures covering 1001–2001. Your IPFS coordinates point to shipwrecks that may or may not actually exist in the vast digital ocean. It’s like an underwater archaeological mystery, except instead of lost civilizations, we’re hunting for lost blockchain data.

So we built Deep Index Dive.

What Does a Deep Sea Data Expedition Look Like?

Deep Index Dive performs what we like to call a “comprehensive deep-sea survey” of your chunk data. Just like a real oceanographic expedition, it uses multiple sonar systems to map the seafloor:

Depth Chart Synchronization

“Do all our navigation instruments agree on where the treasure lies?”

The expedition compares depth charts from three different sonar systems: your manifest (the master navigation chart), your index files (what your equipment actually detected), and your bloom filters (what your fish-finder thinks is down there). In a perfect underwater world, these would all point to the same coordinates 100% of the time.

In the real depths? Well, let’s just say that 98% agreement means you’re probably not going to crash into any underwater mountains, and anything below 95% means someone’s compass is seriously off.

Treasure Chest Inventory

“Are the artifacts as valuable as the treasure map claims?”

Your manifest says that sunken chest contains 16,821,332 bytes of precious blockchain artifacts. When you crack it open? Nothing but sand and seaweed. This happens more often than you’d think in digital archaeology, and it’s usually a sign that the treasure got scattered by underwater currents or that pirates made off with the goods.

Deep Index Dive catalogues every single artifact against what the treasure map claims should be there. When we find discrepancies, we document them with the sort of meticulous detail that would make a real marine archaeologist proud.

Deep Ocean Network Survey

“Can we actually reach the sunken treasures when we need them?”

This is our equivalent of checking if the underwater ruins are actually accessible or if they’ve been swallowed by an oceanic trench. Your manifest lists IPFS coordinates for all your data treasures, but are those coordinates actually reachable in the vast digital ocean?

This survey is optional (use the --check_hashes flag) because it's slow—we're literally sending diving expeditions out to thousands of different coordinates and asking "hey, is there actually treasure here?" But when you need to know if your data is lost at sea, you really need to know.

Ancient Protocol Compatibility

“Are these artifacts from the same civilization?”

This is where things get archaeologically fascinating. Every index and bloom artifact has inscriptions that tell us which version of the civilization (chifra) created it. Our digital archaeologists examine these inscriptions to determine if they’re compatible with current expedition equipment.

Why does this matter? Because blockchain civilizations evolve, and an artifact created by chifra civilization v0.40.0 might not be fully readable by modern v2.0.0 expedition tools. It’s not necessarily lost forever, but it’s worth cataloging for future expeditions.

The Multi-Ocean Digital Explorer

Here’s where things get exciting. Deep Index Dive doesn’t just explore Ethereum’s main ocean — it can survey every EVM-based digital sea that TrueBlocks has charted. The Gnosis archipelago, the Polygon reef system, you name it.

# Survey the main ocean depths
./deepIndexDive

# Explore the Gnosis archipelago  
./deepIndexDive --chain gnosis

# Chart the Polygon reefs and update your navigation first
./deepIndexDive --chain polygon --update some

Each ocean gets its own navigation charts and its own expedition reports. Because mixing up underwater territories is bad for digital archaeology, just like it’s bad for real ocean exploration.

Reading the Expedition Report (Or: How to Interpret Your Digital Oceanography)

Deep Index Dive generates what we call an “Expedition Summary” — a consolidated report of all the underwater surveys it performed. Here’s what a pristine digital ocean looks like:

====================================================================================================================
                                                 EXPEDITION SUMMARY
====================================================================================================================
Survey Mission             Total  Success   Failed    Percent Notes
--------------------------------------------------------------------------------------------------------------------
Depth Chart Sync           5040     4946       94     98.1% Minor chart discrepancies
Treasure Inventory         4946     4946        0    100.0% All artifacts accounted for
Ocean Network Survey       4946     4921       25     99.5% Few remote coordinates unreachable
Protocol Compatibility     4946     4946        0    100.0% All from current civilization
Deep Ocean Index Check        1        1        0    100.0% Core survey successful
--------------------------------------------------------------------------------------------------------------------
OVERALL EXPEDITION        24879    24760      119     99.5%
====================================================================================================================

That’s a pristine digital ocean. Crystal clear waters with all treasures accounted for.

But sometimes you get expedition reports that look more like this:

Survey Mission             Total  Success   Failed    Percent Notes
--------------------------------------------------------------------------------------------------------------------
Depth Chart Sync           5040     4850      190     96.2% Navigation equipment needs calibration
Treasure Inventory         4946     3201     1745     64.7% Many empty treasure chests
Ocean Network Survey       4946     2103     2843     42.5% Severe underwater communication issues

That’s a digital ocean that’s been hit by some serious underwater storms. Time to send in the salvage crews.

The Philosophy of Deep Ocean Exploration

You might ask: “Why do you need a tool that explores the depths of your data? Shouldn’t digital treasures just stay where you put them?”

And you’d be asking the exact right question.

In a perfect world, data would never drift on unknown currents, files would never sink into digital trenches, networks would never fail in the deep waters, and software would never have bugs. But we don’t live in that perfect realm. We live in a world where mysteries accumulate in the depths, where:

Storage drives corrode like old shipwrecks
Network connections get tangled in digital kelp forests
Software has bugs (yes, even our deep-sea exploration gear)
IPFS treasure coordinates sometimes point to empty water
Cosmic rays occasionally scramble our sonar readings

Deep Index Dive embodies what we call “comprehensive oceanographic survey” — the idea that you should never trust digital waters without mapping the depths, and you should resurvey regularly. The data may drift on currents we cannot see.

Diving from YOUR Submarine

Here’s the beautiful part: Deep Index Dive operates entirely from your personal submarine. It doesn’t need to surface and radio some distant marine research station. It doesn’t require you to upload your treasure maps to a floating research platform. It works with your local TrueBlocks diving equipment, using your charts, in your territorial waters.

This isn’t just a philosophical stance about digital sovereignty (though it is that too). It’s a practical necessity. If you’re going to be serious about underwater blockchain archaeology, you need to be able to conduct your own expeditions, using diving gear you control, operating from vessels you trust.

The Art of Digital Marine Archaeology

Deep Index Dive saves detailed expedition logs to ocean-specific archives. Want to know exactly which depth charts disagreed and why? Check ./expeditions/mainnet/depth_chart_sync.txt. Want to see every single missing treasure? Look at ./expeditions/mainnet/treasure_inventory.txt.

Each expedition log includes timestamps, survey statistics, and detailed discovery lists. Because when you’re doing underwater archaeology, you need more than just “something’s down there” — you need to know exactly what’s down there, when it went missing, and how to salvage it.

Support Our Continued Deep Sea Exploration

Deep Index Dive is part of the TrueBlocks project, which remains self-funded and stubbornly independent. If you find this tool useful — or if you just appreciate the idea of being able to conduct comprehensive blockchain archaeology from your own digital submarine — consider supporting our underwater expeditions.

You can donate to our GitCoin grant or send ETH directly to trueblocks.eth. Even small amounts help us keep building tools that let you own and explore your own blockchain ocean depths.

Because at the end of the day, the blockchain may be trustless, but your digital treasures shouldn’t vanish into the sublime depths unnoticed.

Deep Index Dive is available in the /examples/chunkDiagnostics directory of the TrueBlocks Core repository. Build it, launch your expedition, and discover what treasures and mysteries lurk in the depths of your blockchain data.

Got questions? Found bugs? Want to contribute? Join us on Discord or submit issues on GitHub.