TrueBlocks — Final Report for EF Grant FY22–0772

Introduction

9 min readAug 23, 2023

We are pleased to submit our fourth and final progress report to the Ethereum Foundation.

One year ago this month, we were awarded a grant from the Ethereum Foundation to support our work on TrueBlocks. The Foundation’s financial (and non-financial) support has been invaluable. We’ve made significant progress — hitting most of our milestones — culminating in the just-announced release of TrueBlocks v1.0.0.

In short, we are starting to realize the objective we expressed in our original proposal:

…curbing the dominance of today’s indexers and the vulnerabilities they subject [the entire Ethereum community] to. We hope to do this by building a truly decentralized indexing solution ensuring the “address index,” and by extension, blockchain data remains a true public good…

In this final report (which we submit in fulfillment of our grant agreement and in the hopes of receiving our final installment), we will:

Benchmark ourselves on realized vs. expected milestones,
Provide information on our community engagement and usage,
Discuss our future work and sustainability roadmap.

Realized vs. Expected Milestones

In this section, we update our readers on our roadmap and sub-projects submitted with our original grant proposal. While there’s more work to do, overall, we are pleased with all that our two-person team has accomplished.

We’ve made significant progress on two key fronts addressed in the roadmap:

releasing version 1.0.0 (a full GoLang port) and,
raising awareness and community engagement.

The table below shows the massive speedups we’ve achieved by moving to GoLang. This was the bulk of our work. Our version 1.0.0 is feature-complete and will have no further backward-incompatible changes without a migration path.

The Unchained Index (spec) is core to our work. Over the last year, we’ve completed all the fundamental work needed to produce the index and share it exactly how we envisioned it. We gave a presentation this spring discussing this aspect of our work. In the coming weeks, we will finish version 1.0.0 of the spec.

Part of our grant proposal included connecting with other teams and staying informed about other chains. We did a lot of this over the last year (see below). This work continues.

In our original proposal, we expressed an interest in improving our community’s engagement, fostering user acquisition, and increasing the growth in our user base (see below for a discussion of this progress).

Having now finished version 1.0.0, we are excited to focus on engaging more with users. This has already started. Over the next few months, we have four speaking engagements already lined up: ProtocolBerg and dAppCon in Berlin in mid-September, an academic conference in Munich in late September, and, hopefully, some sessions at DevConnect in Istanbul in November.

The more people hear us talk about our work, the more converts we get.

As previously discussed, an always-present objective of TrueBlock has been to get “closer to the node.” We feel the node software is deficient and needs improvement, as it lacks proper address indexing. We have seen progress on this front as well.

A number of our community members have begun sharing our work with others, including node software teams such as Erigon and Reth.

Here’s a bug report written by one of our users pointing out that TrueBlocks finds errors in the node software’s tracing routines: Trace Compatibility with TrueBlocks
In this issue, written against Reth, one of our users is checking the temperature of interest in a new RPC endpoint supporting our chifra list endpoint.
Here are three potential improvements from users suggesting our work should be added to the node’s implementations: a specification for “address appearance”, a specification for a new RPC endpoint (address_getAppearance), and another spec for another RPC endpoint (eth_getAddressesInBlock). Our work on TrueBlocks heavily influenced this work, as noted earlier.

We’ve also had numerous conversations over the last year with many groups, including Test In Production, who’s writing an Erigon clone for the Optimism Stack, Infura (long-time friends), people from UniSwap, Yearn, ScopeLift, Rotki, Hopr Network, Giveth, GitCoin, and many others. People who know our work know it’s important. Our task remains to get more people to learn about our work.

The GoLang Port

The GoLang port took longer than we expected, but it was well worth it. Since July, all of our C++ is not only gone but gone forever. Good riddance. While it served us well over the years, the C++ code was long past retirement age.

We’re very excited to share our GoLang work with others. GoLang makes building re-usable packages WAY easier than C++. Also, GoLang, being a modern language, tends to attract more contributors. We’ve definitely seen an increase in contributors over the last year. It’s also very easily documented.

Plus — and this is super important — GO IS FAST.

Here’s a chart comparing our previous C++ code with the GoLang code. Green is good. Yellow is better. Red will be fixed soon. It’s not perfect, but pretty many of them are yellow: a ten (10) times or more speedup.

Community Engagement & Usage

Being open-source, it’s difficult to know exactly how widely our software is being used. We’re frequently surprised by mentions on social media from people who we’d never heard of using our software.

Because our software works best when running directly against a locally installed node, we’ve always expected adoption to be slow. We’ve felt that until the node is more useful and easier to install, folks simply won’t run nodes. Reth and packages like dAppNode make the node easier to install. TrueBlocks makes the node more useful. It’s a bit of a chicken-and-egg problem.

However, as regulation, government sanctioning, whitelisting, and improved tooling (such as TrueBlocks) become available, we will see growth in usage. In the past year, we saw a huge surge in interest in TrueBlocks soon after the US Government sanctioned Tornado Cash. When censorship rears its head, people seek out truly decentralized solutions.

Other positive trends in the past year…

GitCoin and Giveth engaged TrueBlocks in a consulting capacity as they began exploring a more decentralized data infrastructure in the hopes of becoming more self-sovereign. Both organizations are struggling to access on-chain data in a cost-effective, cheap, and meaningful way.

Uniswap Foundation invited us to submit a grant proposal to explore how TrueBlocks might help them (as they put it) “decentralize their data infrastructure.” They told us they searched the entire ecosystem and approached only a few teams with such an invitation. We’re still waiting for a final decision.

In the second Optimism Retroactive Public Goods Funding Round, TrueBlocks was the 18th largest recipient of funding among 43 infrastructure projects (and 195 projects in total). This was a very welcome validation of our work. We plan to continue to engage with the Optimism community.

On Docker Hub, we’ve seen nearly 4,000 downloads of our TrueBlocks docker version, which we released in beta about four months ago, and version 1.0.0 this month. We’re still investigating how to get more detailed information about the usage of our docker version.

Membership in our Discord channel has more than doubled in the past year, and engagement from the developer community is growing (GitHub forks, stars, and contributors demonstrate this). Our GitHub stars have continued to grow despite the bear market:

A few of our discord members have sought support as they try to index their own EVM chains (using the Unchained Index smart contract) showing that our ideas are not only able to handle cross-chain but are needed there as well. This happens on mainnet, alternative layer ones, and numerous layer twos. Our users’ questions and interactions continue and have certainly informed our research efforts in the Layer 2 space.

As previously mentioned, our work has inspired some EIPs as well.

Links and Twitter Mentions

Here’s a smattering of links to public Twitter tweets during the May-August 2023 period that demonstrate community views on TrueBlocks. A more thorough list is here.

One developer understands exactly what we’re shooting for: https://twitter.com/moo9000/status/1671294395398717442.

Here’s a user expressing his admiration for TrueBlocks:

And another calling us amazing…

We’ve pulled many other tweets over the past few months highlighting the favorable view and increasing recognition of TrueBlocks.

Future Work

I sometimes tell people that there is nothing anyone can do to make me stop working on TrueBlocks (well, within reason, I guess), and this is true. We are encouraged by the support we’ve gotten over the past year. Here is a list of what we intend to work on in the coming year:

Further improvement and optimization of software, including the user and developer experience. Faster speed, smaller footprint, and good UX are the hallmarks of usable software.
Refine our toolkits and SDKs to make it easier for others to leverage our work. For example, better testing scripts, expansion of the ideas behind the Unchained Index, better documentation, and better usability are envisioned.
Educate through conference talks, video tutorials, articles, explainers, and other means about why TrueBlocks is built the way it is (local-first) , why it’s important, and the amazing benefits one sees from truly local software.
Be more systematic about community and user engagement, eliciting more and better feedback and input.
Create what we call “mini-dApps”, which are fully local, (truly) decentralized applications running on end users’ machines. This will be the easiest way to demonstrate the unique nature of our work.
Work with the node software teams to either (a) implement indexing in the node (possibly optionally on startup) or (b) work on “extensions” to the node (in both the general sense of making it possible to extend the node, and the particular sense in making indexing a part of the node).
Go deeper into our exploration of how one might index layer twos. Particularly as this relates to the Optimism Stack, of which we are huge fans.
Complete version 1.0.0 of the Specification for the Unchained Index. This work is complete in the code and only needs to be documented.
Explore writing a Reth extension, creating an optional feature on the node for doing indexing. Reth has invited us to do so, and we hope to work on this soon. This would make it that much easier for end users to get an actually useful node that can effectively serve applications.
Seek additional (and continued — hint, hint) funding from as many sources as possible.

One Final Possible Idea

This is just an idea I’ve been toying with…

TrueBlocks has nearly 2,000 test cases that we’ve accumulated over the years. These test cases are unique. Since 2016, each time there was a bug in one of the pieces of node software, or a hard fork, or some other unusual occurrence (such as the switch to POS or the DOA hack), we wrote test cases surrounding it.

This means that our test cases find edge cases.

Also, our indexing scraper touches every trace in every block.

We envision a continually-running test system that node developers could sign up for that would immediately identify reversions to their RPC and/or help bring all nodes’ RPC endpoints into closer alignment.

Conclusion

The support from the Ethereum Foundation has helped us build a truly decentralized indexing solution. We’ve completed version 1.0.0, which means everything is working, and we’re feature-complete.

While we’ve made great strides over the year, as with any open-source public infrastructure good, there’s always more work to be done.

We hope the Ethereum Foundation (or anyone else who’s reading this) will help us by providing further funding. We trust you’re convinced we’ve used your most generous grant money well.

Thank you so much, and Cheers!

Thomas Jay Rush
Philadelphia, PA, USA
August 23, 2023