TrueBlocks Comparison with Alchemy, Covalent, and Etherscan

Or How I Learned to Love Local-First Indexing

The Problem

It’s impossible to do perfect, automated, off-chain state reconciliation using blockchains.

This absurd fact is true even though, internally, blockchains reconcile the state of hundreds of millions of accounts to 18-decimal-place accuracy every 12 seconds.

Introduction

This article compares the Unchained Index with three major blockchain data providers: Alchemy, Covalent, and Etherscan. The results may surprise you.

Spoiler alert: The Unchained Index (TrueBlocks) outperforms them all.

The Unchained Index is the best index of appearances available. It indexes every appearance of every address anywhere on any chain. No other indexer does this.

This article explains why we feel comfortable making this equally absurd claim.

Our Experiment

We queried 1,000 randomly selected addresses for account histories using TrueBlocks, Alchemy, Covalent, and Etherscan. The results are presented below.

We wanted to conduct a fair test, so first, we carefully read each provider’s API documentation and tried to use their endpoints correctly. Next, we made sure our code handled error responses correctly. We chose an easy path. If there were any errors, we restarted the tests from scratch.

Side Note

Because TrueBlocks is local-first and not a “shared resource” like the other providers, it exhibits no rate limiting. Therefore, it is MUCH faster than the three alternative providers. TrueBlocks outperforms Web 2.0 APIs because its faster. It can dig deeper into the data.

The Results

As noted, we tested 1,000 randomly selected addresses against four different providers. The list of addresses is available in addresses.txt. The README explains how to run the tests for yourself.

Preliminary Cleaning

Of the 1,000 addresses, 102 had no transactions; therefore, these addresses were discarded.

Additionally, we found 207 addresses that transacted 5,000 or more times. We discarded these addresses as well. We did this out of a sense of fairness. Some APIs (Etherscan) limit the overall number of transactions they will return. We kept far below these limits; otherwise, TrueBlocks would have outperformed even more drastically than it did. Also, the other systems are so slow that these large addresses take too long.

This left us with 691 addresses to test. The results are summarized below.

* — See Bug in Etherscan for an explanation of this number.

What does this data mean?

• Initial addresses included is the initial number of addresses before filtering data cleaning.
• Eliminated due to > 5,000 is the number of addresses that had 5,000 or more transactions. These addresses were removed.
• Eliminated due to zero transactions is the number of addresses for which the given provider returned no transactions.
• Addresses queried is the total number of addresses we report on in our results.
• Appearances found are the total number of appearances of an address returned by a provider.
• Addresses with transactions only found by… are the number of addresses for which the given provider was the only provider that returned appearances for the address.
• Unique appearances not found by others are the number of appearances found by the given provider but not others.
• Unique appearances where balance changed shows how many Unique Appearances were materially important (i.e., involved ETH balance changes).

Summary

Of the 691 addresses that were queried for all providers:

• 506 (73%) had appearances that TrueBlocks found but no other provider found. That’s 120,744 more appearances!
• 1,194 of those appearances were “material” (i.e., the address’ ETH balance changed).
• Zero appearances were found by other providers that were not found by TrueBlocks.
• Only TrueBlocks found appearances for every address.
• For 14 addresses, Etherscan found appearances that TrueBlocks did not find, but in all 14 cases, the difference was due to a bug in Etherscan. (See below.)

We recognize that these results are surprising and may seem like a mistake. However, one must realize that TrueBlocks looks for more than just a small set of known behaviors (such as Transfers easily found in Events). TrueBlocks looks everywhere. In particular, TrueBlocks looks in:

• the obvious places (to, from, log generator, etc.)
• every transaction’s input data
• every log’s topics and data
• every trace’s input and output fields
• mining and uncle rewards
• staking withdrawals

Here’s the breakout of where those 456,269 appearances were found:

Bug in Etherscan related to Uncles

For 14 of the 687 addresses searched by Etherscan, appearances were returned that TrueBlocks did not find. In every case, however, the difference was due to a bug in Etherscan related to uncles. The bug is that Etherscan returns the block number when the uncle was “produced.” TrueBlocks returns the block number in which the uncle reward was credited to the miner’s account. We know this because we ran the following analysis on all 364 appearances of this issue.

First, we extracted just the block number from the appearances found by Etherscan. We then calculated 1 block prior to that block number (P) and seven blocks after that block number (A). We then ran:

chifra state --parts balance P-A <address> --changes

which uses another one of the chifra tools to extract the balances for the given address at the given blocks. For example, for address 0x3f98e477a361f777da14611a7e419a75fd238b6bEtherscan reports the following appearances:

485,uncle
940,uncle
1114,uncle
...

This command

chifra state --parts balance 484-492 0x48040276e9c17ddbe5c8d2976245dcd0235efa43

returns

As you can see, Etherscan reports the uncle block at block 485. However, the uncle's reward was not credited to the miner’s account until the block 487. TrueBlocks reports the uncle at block 487.

In all 364 cases, the block Etherscan reports as the uncle block is technically correct. However, the uncle reward was not credited to the miner’s account until a few blocks later. In each case, that block was the block that TrueBlocks reported. Etherscan got it wrong. Unless you want to lean on a technicality. I would argue that a change in the balance of an account is the correct place to note in an address’s history. I’ll leave it up to Etherscan to decide if they want to fix this bug.

The total number of places where Etherscan legitimately found more appearances than TrueBlocks: Zero!

What is an Appearance?

“Appearances” are seemingly simple. For any address, the address’s list of appearances is a list of blocknumber.transactionId pairs, noting wherever the address appears on the chain.

For example, the first three appearances of trueblocks.eth are:

Easy enough. Just look at from, to, log topics and a few other places. That's what most indexers do. But as we've demonstrated above, there's way more to the story. Please see the very detailed discussion on the Specification of the Unchained Index.

Why Does This Matter?

Blockchains are perfect, 18-decimal place accurate accounting systems. Every 12 seconds, they come to balance on many hundreds of millions of accounts. That’s a fact of nature.

The fact that even Etherscan, our industry’s leading data provider, can’t get this right is important because blockchains should balance perfectly off-chain and on-chain.

What the hell are we even building if we can’t account for every single wei for every single account? What have we built?

I claim we’ve built nothing of interest. Convince me otherwise!

Replicating The Test

Staying objective and working on permissionless public goods has always been a primary concern. As with the trueblocks-core repos, which require no trust between the parties (us, our users, data consumers, you!), we want to be sure anyone can replicate these tests. See the README linked above.

Share your results with us, on X/Twitter or GitHub. If you have questions, we invite you to join our Discord server.

Conclusion

TrueBlocks is funded from personal funds and grants from The Ethereum Foundation (2018, 2022, 2024), Optimism Retro PGF (2022, 2023), Consensys (2019), Moloch DAO (2021), Filecoin/IPFS (2021), and our lovely GitCoin donors.

If you like this article and wish to support our work, please donate to our GitCoin grant using ETH or any token. If you’re an Optimism badge holder, vote for our project. Or send us a tip directly at trueblocks.eth or 0xf503017d7baf7fbc0fff7492b751025c6a78179b.