By: Vitalik Buterin
Compiled by: Glendon, Techub News
One of the challenges with Ethereum is that, by default, any blockchain protocol will grow in bloat and complexity over time. This happens in two places:
Historical data: Any transaction made and any account created at any point in history needs to be permanently stored by all clients and downloaded by any new client to be fully synchronized to the network. This causes the client load and synchronization time to increase over time, even if the capacity of the chain remains the same.
Protocol features: It is much easier to add new features than to remove old ones, which causes code complexity to increase over time.
For Ethereum to be sustainable in the long term, we need to exert strong counter-pressure to both trends, reducing complexity and bloat over time. But at the same time, we need to preserve one of the important properties that makes blockchains great: Permanence . You can put an NFT, a love letter in Transaction Calldata, or a smart contract with a million dollars on the chain, then go into a cave for ten years and find it is still there waiting for you to read and interact with it. For DApps to be fully decentralized and remove upgrade keys, they need to be confident that their dependencies will not be upgraded in a way that breaks them - especially Layer 1 itself.
The Purge, 2023 Roadmap
Balancing these two needs, and minimizing or reversing bloat, complexity, and decay while maintaining continuity, is entirely possible if we put our minds to it. Organisms can do this: while most age over time, a lucky few do not . Even social systems can have extremely long lifespans . Ethereum has already succeeded in some cases: proof of work is gone, the SELFDESTRUCT opcode has largely disappeared, and beacon chain nodes already store up to six months old data. Figuring out this path for Ethereum in a more general way, and moving toward a long-term stable end result, is the ultimate challenge for Ethereum's long-term scalability, technical sustainability, and even security.
The Purge: Key Goals
Lower client storage requirements by reducing or eliminating the need for each node to permanently store all history or even final state.
Reduce protocol complexity by eliminating unnecessary functionality.
History expiry
What problem was solved?
As of this writing, a fully synchronized Ethereum node requires about 1.1 TB of disk space for the execution client , plus hundreds of GB for the consensus client. The vast majority of this is history: data about historical blocks, transactions, and receipts, much of which is many years old. This means that even if the gas limit did not increase at all, the size of the node would increase by hundreds of GB per year.
What is it and how does it work?
A key simplifying feature of the history storage problem is that since each block points to the previous block via hash links (and other structures ), consensus on the present is sufficient to reach consensus on the history. As long as the network agrees on the latest block, any historical block or transaction or state (account balance, random number, code, storage) can be provided by any single participant with a Merkle proof that allows anyone else to verify its correctness. While consensus is an N/2-of-N trust model, history is a 1-of-N trust model .
This leaves us with many choices for how we store historical data. A natural choice is a network where each node stores only a small fraction of the data. This is how torrent networks have worked for decades: while the network stores and distributes millions of files in total, each participant stores and distributes only a few of them. Perhaps counterintuitively, this approach doesn’t even necessarily reduce the robustness of the data. If, by making nodes cheaper to run, we can get a network of 100,000 nodes where each node stores a random 10% of the history, then every piece of data will be replicated 10,000 times — exactly the same replication factor as a 10,000 node network, where each node stores everything .
Today, Ethereum has begun to move away from the model where all nodes permanently store all history. Consensus blocks (that is, the part related to proof-of-stake consensus) are only stored for about 6 months. Blobs are only stored for about 18 days. EIP-4444 aims to introduce a one-year storage period for historical blocks and receipts. The long-term goal is to have a coordinated period (probably about 18 days) during which each node is responsible for storing everything, and then build a peer-to-peer network of Ethereum nodes to store old data in a distributed manner. (Techub News Note: Blob is a data storage concept designed to reduce the transaction costs of L2 expansion solutions.)
Erasure codes can be used to improve robustness while keeping the replication factor the same. In fact, blobs already use erasure codes to support data availability sampling. The simplest solution might be to reuse this erasure code and put the execution and consensus block data into blobs as well.
What are the connections with existing research?
Portal network
What else needs to be done and what trade-offs need to be made?
The main remaining work involves building and integrating a concrete distributed solution for storing history - at least execution history, but eventually consensus and blobs as well. The simplest solutions to this are (i) simply bringing in existing torrent libraries, and (ii) an Ethereum-native solution called the " Portal Network ". Once either of these are brought in, we can enable EIP-4444. EIP-4444 itself does not require a hard fork, but it does require a new network protocol version. Therefore, it is worth enabling it for all clients at the same time, otherwise there will be a risk of failure when clients connect to other nodes that expect to download the full history but are unable to do so.
The main trade-off involves how hard we try to make “ancient” historical data available. The simplest solution is to stop storing ancient history tomorrow and rely on existing archive nodes and various centralized providers to replicate it. This is easy, but it weakens Ethereum as a permanent record place. The more difficult but safer approach is to first build and integrate a torrent network to store history in a distributed way. Here, “how hard we try” has two dimensions:
1. How do we try to ensure that the largest set of nodes actually stores all the data?
2. How do we deeply integrate history storage into the protocol?
For (1), a most paranoid approach is to use proof of custody : effectively requiring each proof-of-stake validator to store a certain percentage of the history, and periodically cryptographically checking that they are doing so. A more moderate approach would be to set a voluntary standard for what percentage of history each client stores.
For (2), the basic implementation only involves work that is already done today: Portal already stores ERA files containing the entire history of Ethereum. A more thorough implementation would actually require connecting this to the sync process, so that if someone wants to sync a full history storage node or archive node, they can do so by syncing directly from the Portal network, even if no other archive nodes exist online.
How does it interact with the rest of the roadmap?
If we want to make it extremely easy to run or spin up a node, then reducing history storage requirements is arguably more important than statelessness: of the 1.1 TB required for a node, ~300 GB is state storage, and the remaining ~800 GB is history storage. The vision of an Ethereum node running on a smartwatch and taking only minutes to set up is only possible if both statelessness and EIP-4444 are achieved.
Limiting historical storage also makes it easier for newer Ethereum node implementations to only support the latest version of the protocol, making them simpler. For example, many lines of code can now be safely removed because the empty storage slots created during the 2016 DoS attack have all been deleted . Clients can safely remove all proof-of-work related code now that the move to proof-of-stake is history.
State expiry
What problem does it solve?
Even if we eliminated the need for clients to store history, client storage requirements would continue to grow, by about 50 GB per year, as the state continues to grow: account balances and nonces, contract code, and contract storage. Users could pay a one-time fee that would forever burden current and future Ethereum clients.
State is harder to "expire" than history, because the EVM is fundamentally designed around the assumption that once a state object is created, it will exist forever and can be read by any transaction at any time. If we introduce statelessness, some people think that this problem may not be so bad: only a specialized class of block builders need to actually store state, and all other nodes (even list generation) can run statelessly. However, there is an argument that we don't want to rely too much on statelessness , and ultimately we may want to let state expire to keep Ethereum decentralized.
What is it and how does it work?
Right now, when you create a new state object (which can be achieved in one of three ways: (i) sending ETH to a new account; (ii) creating a new account using code; or (iii) setting up a previously untouched storage slot), that state object will stay in that state forever. Instead, what we want is for objects to automatically expire over time. To do this, the key challenge is to achieve the following three goals:
1. Efficiency : No need for a lot of additional computation to run the expiration process;
2. User-friendliness : If “someone goes into a cave and comes back five years later,” they shouldn’t lose access to their ETH, ERC20, NFT, CDP positions.
3. Developer friendliness : Developers do not have to switch to a completely unfamiliar mental model. In addition, currently rigid and unupdated applications should continue to work normally.
In situations where these goals are not met, the problem is easily solved. For example, a user could have each state object also store a counter to represent its expiration date (the expiration date can be extended by burning ETH, which can happen automatically when reading or writing), and have a loop "walking through the state" to remove expired state objects. However, this introduces additional computation (and even storage requirements), and it certainly does not meet the requirement of user-friendliness. It is also difficult for developers to reason about edge cases involving stored values sometimes reset to zero. If users set expiration timers contract-wide, this makes the developer's job technically easier, but economically more difficult: the developer must consider how to "pass on" the ongoing storage costs to the user.
These are issues that the Ethereum core development community has been working on for years, including proposals such as " Blockchain Rent " and " Regenesis ". Ultimately, we combined the best parts of these proposals and focused on two categories of "least-worst known solutions":
Partial Status Expiration Solution
State expiration recommendations based on address cycles.
Partial status expired
The partial state expiration proposals all follow the same principle. We split the state into multiple blocks. Everyone permanently stores a "Top-Level Map" with blocks that are either empty or non-empty. The data in each block is only stored if it was recently accessed. There is a "Resurrection" mechanism, where if a block is no longer stored, anyone can recover the data by providing proof of the data.
The main differences between these proposals are: (i) how do we define "recent", and (ii) how do we define "large chunks of data"? One specific proposal is EIP-7736 , which builds on the "stem and leaf" design introduced for Verkle trees (although compatible with any form of stateless tree, such as binary trees). In this design, headers, code, and storage slots that are adjacent to each other are stored under the same "stem". The data stored under the "stem" is at most "256 * 31 = 7936 bytes". In many cases, the entire header and code of an account and many key storage slots will be stored under the same "stem". If the data under a given "stem" has not been read or written within 6 months, the data is no longer stored, and only a 32-byte commitment to the data (a "stub") is stored. Future transactions that access that data will need to "resurrect" the data and provide a proof of verification against the "stub".
There are other ways to implement similar ideas. For example, if account-level granularity is not enough, we could make a scheme where each "1/2 to the power of 32" part of the "tree" is controlled by a similar "stem and leaf" mechanism.
This is made more tricky because of incentives: an attacker could force clients to store large amounts of state permanently by putting a large amount of data into a single subtree and sending a single transaction every year to "update the tree." If the update cost is made proportional to the size of the tree (or inversely proportional to the update duration), then someone could potentially harm other users by putting a large amount of data into the same subtree as them. Users can try to limit both of these issues by making the granularity dynamic based on the subtree size: for example, each consecutive "2 to the power of 16" = 65536 state objects can be considered a "group." However, these ideas are more complex; the "stem" based approach is simple, and it aligns with incentives because typically all data under the "stem" is related to the same application or user.
Address cycle based state expiry recommendations
What if we want to completely avoid any permanent state growth, even with a 32-byte stub? This is a hard problem due to resurrection conflicts : if a state object is deleted, and a later EVM execution puts another state object in the exact same place, but then someone who cares about the original state object comes back and tries to restore it? In the case of partial state expiration, the stub prevents new data from being created. In the case of full state expiration, we can't even store the stub.
The address cycle based design is the best way to solve this problem. Instead of using one state tree to store the entire state, we have a growing list of state trees, and any state read or written is saved in the latest state tree. A new empty state tree is added once every cycle (e.g. 1 year). Older state trees will remain stable. Full nodes should only store the two most recent state trees. If a state object has not been touched in two cycles and therefore falls into an expired state tree, it can still be read or written, but the transaction needs to prove its Merkle proof - once proven, a copy will be saved in the latest state tree again.
A key idea that makes this all user- and developer-friendly is the concept of address cycles. An address cycle is a number that is part of an address. The key rule is that an address with address cycle N can only be read from or written to during or after cycle N (i.e., when the state tree list reaches length N) . If a user wants to save a new state object (e.g., a new contract or a new ERC20 balance), if you make sure you put the state object into a contract with address cycle N or N-1, then you can save it immediately without providing proof that there was nothing before. On the other hand, any additions or edits to state in an old address cycle require proof.
This design retains most of the current features of Ethereum, with very little additional computation, allowing applications to be written almost as they are now (ERC20 needs to be rewritten to ensure that the balance of an address with an address cycle of N is stored in a subcontract that itself has an address cycle of N), and solves the "users in the cave for five years" problem. However, it has a big problem: the address needs to be expanded to more than 20 bytes to accommodate the address cycle .
Address space extension
One proposal is to introduce a new 32-byte address format that includes a version number, address period number, and extended hash value.
The red one is the version number. The four orange zeros represent the empty space that can accommodate the shard number in the future. The green one is the address cycle number. The blue one is the 26-byte hash value.
The key challenge here is backward compatibility. Existing contracts are designed around 20-byte addresses, and often use strict byte packing techniques that explicitly assume addresses are exactly 20 bytes long. One idea to address this is to use a translation map, where legacy contracts interacting with new-style addresses will see the 20-byte hash of the new-style address. However, there is significant complexity in ensuring this is secure.
Address space contraction
Another approach goes in the opposite direction: we immediately ban some "2 to the power of 128" address subrange (e.g., all addresses starting with 0xffffffff), and then use that range to introduce addresses with address periods and 14-byte hash values.
The main sacrifice of this approach is that it introduces security risks for counterfactual addresses : addresses that hold assets or permissions but whose code has not yet been published to the chain. The risk is that someone creates an address that claims to own a piece of (yet unpublished) code, but there is another valid piece of code that hashes to the same address. Computing such a collision currently requires 2 to the power of 80 hashes; address space shrinkage would reduce this number to an easily accessible 2 to the power of 56 hashes.
The key risk area is counterfactual addresses, i.e. wallets that are not held by a single owner, which is a relatively rare situation today but will likely become more common as we move into a multi-L2 world. The only solution is to simply accept this risk but identify all common use cases where it could go wrong and come up with effective workarounds.
What are the connections with existing research?
Early Proposals
Ethereum state size management theory
Several possible ways to cause statelessness and state expiration
Partial state expiration proposal: EIP-7736
Address Space Extension Documentation:
What else needs to be done and what trade-offs need to be made?
I see four possible paths forward:
1. We are stateless, and never introduce state expiration . State is growing (albeit slowly: we probably won’t see it exceed 8 TB for decades), but only needs to be held by a relatively specialized class of users: not even PoS validators need state.
One feature that requires access to part of the state is inclusion list generation, but we can do this in a decentralized way: Each user is responsible for maintaining the portion of the state tree that contains their own account. When they broadcast a transaction, they broadcast proofs of the state objects that were accessed during the verification step (this works for both EOA and ERC-4337 accounts). The stateless validator can then combine these proofs into a proof of the entire inclusion list.
2. We implement partial state expiration and accept a much lower but still non-zero permanent state size growth rate. This result is arguably similar to how history expiration proposals involving peer-to-peer networks accept a much lower but still non-zero permanent history storage growth rate, since each client must store a lower but fixed percentage of history data.
3. We use address space extensions for state expiration. This will involve a multi-year process to ensure that the address format conversion method is effective and safe, including for existing applications.
4. We use address space shrinkage for state expiration. This will involve a multi-year process to ensure that all security risks involving address conflicts (including cross-chain situations) are handled.
The important point is that whether or not a state expiration scheme that relies on address format changes is implemented, the hard problem of address space expansion and contraction must eventually be solved . Currently, it takes about 2^80 hash operations to produce an address collision, and this computational load is feasible for extremely well-resourced actors: GPUs can perform about 2^27 hash operations, so they can calculate 2^27 times in a year, so all about 2^30 GPUs in the world can calculate a collision in about 1/4 of a year, and FPGAs and ASICs can accelerate this process further. In the future, such attacks will be open to more and more people. Therefore, the actual cost of implementing full state expiration may not be as high as it seems, because we have to solve this very challenging address problem anyway.
How does it interact with the rest of the roadmap?
Doing state expiration will likely make transitioning from one state trie format to another easier, since no conversion process is required: you can simply start creating a new state trie with the new format, then do a hard fork to convert the old trie. So while state expiration is complex, it does help simplify other aspects of the roadmap.
Feature cleanup
What problem was solved?
One of the key prerequisites for security, accessibility, and trusted neutrality is simplicity. If a protocol is beautiful and simple, then it is less likely to have bugs. It increases the chances that new developers will be able to participate and use any part of it. It is more likely to be fair and easier to defend against special interests. Unfortunately, protocols, like any social system, will by default become more complex over time. If we don’t want Ethereum to fall into a black hole of growing complexity, we need to do one of two things: (i) stop making changes and ossifying the protocol, or (ii) be able to actually remove features and reduce complexity. A middle path, where fewer changes are made to the protocol and at least a little complexity is removed over time, is also possible. This section will discuss how to reduce or eliminate complexity.
What is it and how does it work?
There is no single big fix that will reduce protocol complexity; the nature of the problem is that there are many small solutions.
One example that is mostly done is the removal of the SELFDESTRUCT opcode , and it can serve as a blueprint for how to approach the other examples. The SELFDESTRUCT opcode is the only opcode that can modify an unlimited number of slots within a single block, requiring clients to implement more complexity to avoid DoS attacks. The original purpose of this opcode was to enable voluntary state clearing, allowing the state size to decrease over time. In practice, few people use it. In the Dencun hard fork, the opcode was weakened to only allow self-destructing accounts created in the same transaction. This solves the DoS problem and allows for significant simplification of client code. In the future, it may make sense to eventually remove this opcode entirely.
Some key examples of protocol simplification opportunities identified so far include: First, some examples outside the EVM; these examples are relatively non-intrusive and therefore easier to reach consensus on and implement in a shorter time frame.
RLP → SSZ conversion: Originally, Ethereum objects were serialized using an encoding called " RLP ". RLP is untyped and unnecessarily complex. Today, the beacon chain uses SSZ , which is significantly better in many ways, including supporting not only serialization but also hashing. Eventually, we hope to get rid of RLP completely and move all data types into SSZ structures, which in turn will make upgrades much easier. The EIPs currently proposed for this include [1] [2] [3] .
Removing old transaction types: There are too many transaction types today, and many of them are likely to be removed. In addition to removing them completely, a more moderate alternative is the account abstraction feature, where Smart Accounts can contain code to handle and verify old-style transactions as needed (if they wish).
LOG Reform: The log creates Bloom filters and other logic that adds complexity to the protocol, but is not actually used by clients because it is too slow. We can remove these features and instead work on finding alternatives, such as out-of-protocol decentralized log reading tools using modern technologies such as SNARKs.
Eventually cancel the Beacon Chain Synchronization Committee mechanism: The synchronization committee mechanism was originally introduced to implement light client verification of Ethereum. However, it increases the complexity of the protocol. Eventually, we will be able to verify the Ethereum consensus layer directly using SNARKs , which will eliminate the need for a dedicated light client verification protocol. By creating a more "native" light client protocol (involving verifying signatures from a random subset of Ethereum consensus validators), perhaps a change in consensus can allow us to cancel the synchronization committee earlier.
Data format unification: Currently, execution state is stored in Merkle Patricia trees, consensus state is stored in SSZ trees , and blobs are committed via KZG commitments . In the future, it makes sense to create a single unified format for block data and state. These formats will meet all important requirements: (i) simple proofs for stateless clients, (ii) serialization and erasure coding of data, and (iii) standardized data structures.
Removing the Beacon Chain Committee: This mechanism was originally introduced to support a specific version of execution sharding . Instead, we ended up sharding via L2 and blobs . As a result, the committee is unnecessary and we are in the process of removing it .
Remove mixed endianness: The EVM is big endian, while the consensus layer is little endian. It might make sense to reconcile and make everything either big endian or little endian (probably big endian, since the EVM is harder to change).
Here are some examples of what's inside the EVM:
Simplify Gas Mechanism: Current Gas rules are not well optimized to give clear limits on the amount of resources required to validate a block. Key examples of this include (i) storage read/write costs, which are intended to limit the number of reads/writes in a block, but are currently quite arbitrary, and (ii) memory padding rules, where it is currently difficult to estimate the maximum memory consumption of the EVM. Proposed fixes include stateless Gas cost changes , unifying all storage-related costs into a simple formula, and memory pricing proposals .
Removing precompiles: Many of the precompiles Ethereum currently has are both unnecessarily complex and relatively rarely used, and account for a large percentage of consensus failures, yet are not actually used by any applications. Two ways to deal with this are to (i) remove the precompile outright, and (ii) replace it with (inevitably more expensive) EVM code that implements the same logic. This draft EIP proposes to do this first for identity precompiles; subsequently, RIPEMD160, MODEXP, and BLAKE may be removed.
Remove Gas observability: Make it so that EVM execution can no longer see the amount of Gas remaining. This will break some applications (most notably sponsored transactions), but will make future upgrades easier (e.g., upgrading to a more advanced multi-dimensional Gas version). The EOF specification already makes Gas unobservable, but to simplify the protocol, EOF needs to be mandatory.
Improvements in static analysis: Today, EVM code is difficult to statically analyze, especially because jumps can be dynamic. This also makes it harder to optimize EVM implementations (precompile EVM code into other languages). We can solve this by removing dynamic jumps (or making them more expensive, e.g., gas cost is linear in the total number of JUMPDESTs in the contract). EOF can do this, but to get the benefits of protocol simplification, EOF needs to be enforced.
What are the connections with existing research?
- Next steps for cleanup
- Self- destruct
- SSZ-ized EIPS: [1] [2] [3]
- Stateless Gas Cost Changes
- Linear Memory Pricing
- Precompiled deletion
- Bloom filter removal A method for off-chain secure log retrieval using incremental verifiable computation (read: recursive STARKs)
What else needs to be done and what trade-offs need to be made?
The main trade-offs in making such feature simplifications are (i) how much and how quickly we simplify vs. (ii) backwards compatibility. The value of Ethereum as a chain is that it is a platform on which users can deploy an application and be confident that it will still work years later. At the same time, it is possible to push this ideal too far and, in the words of William Jennings Bryan , “nail Ethereum to the cross of backwards compatibility.” If there are only two applications in all of Ethereum that use a given feature, and one of them has had no users for years, is almost completely unused, and has gained a total of $57 in value, then we should remove that feature, paying the victim $57 out of pocket if necessary.
The broader societal problem is creating a standardized process for making non-urgent, backwards-compatibility-breaking changes. One way to address this is to examine and extend existing precedents, such as the SELFDESTRUCT process. That process would look like this:
- Step 1: Start a discussion about removing feature X
- Step 2: Perform an analysis to determine the impact of removing X on the application, and based on the results: (i) abandon the idea, (ii) proceed as planned, or (iii) find a modified “least disruptive” way to remove X and proceed.
- Step 3: Make a formal EIP to deprecate X. Make sure popular high-level infrastructure (e.g. programming languages, wallets) respect this and stop using the feature.
- Step 4: Finally, actually remove X
There should be a multi-year process between steps 1 and 4, with clear instructions on which projects are in which step. At this point, there is a trade-off between "the intensity and speed of the feature removal process" and "being more conservative and investing more resources in other areas of protocol development", but we are still far from the "Pareto frontier". (Techub News Note: The Pareto frontier refers to the set of all Pareto optimal solutions in multi-objective optimization problems)
EOF
The main set of changes proposed for the EVM is the EVM Object Format (EOF) . EOF introduces a lot of changes, such as disabling gas observability, code observability (i.e. no CODECOPY), and only allowing static jumps. The goal is to allow more upgrades to the EVM to have more powerful properties while maintaining backward compatibility (because the EVM before EOF still exists).
The benefit of this is that it creates a natural path for adding new EVM features and encouraging migration to a stricter EVM with stronger guarantees. The downside is that it significantly increases the complexity of the protocol unless we can find a way to eventually deprecate and remove the old EVM. A major question is: what role does EOF play in the EVM simplification proposal, especially if the goal is to reduce the complexity of the entire EVM ?
How does it interact with the rest of the roadmap?
Many of the "Improve" suggestions in the rest of the roadmap are also opportunities to simplify older features. To repeat some of the examples above:
Switching to single-slot finality gives us the opportunity to remove committees, reformulate economics, and make other proof-of-stake related simplifications.
By fully implementing account abstraction, we can remove a lot of the existing transaction processing logic and move it into a piece of "default account EVM code" that all EOAs can replace.
If we move the Ethereum state to a binary hash trie, this can be reconciled with a new version of SSZ so that all Ethereum data structures can be hashed in the same way.
A more radical approach: converting large parts of the protocol into contract code
A more radical strategy for simplifying Ethereum is to keep the protocol intact but move much of it away from protocol functionality and into contract code.
The most extreme version would be to make Ethereum L1 “technically” just the beacon chain, and introduce a minimal VM (such as RISC-V , Cairo , or a simpler VM specialized for proof systems) that allows others to create their own Rollups. The EVM would then be the first of these Rollups. Ironically, this is exactly the same outcome as the 2019-20 execution environment proposal , although SNARKs make it more feasible to actually implement.
A more modest approach is to keep the relationship between the beacon chain and the current Ethereum execution environment unchanged, but do an in-place swap of the EVM. We could choose RISC-V, Cairo, or another VM as the new "official Ethereum VM", and then force all EVM contracts to be converted to the new VM code (either by compilation or interpretation) that interprets the logic of the original code. In theory, this could even be done with the "target VM" as a version of EOF.
