A Primer to Flashbots and MEVs – The Dark Forest of Ethereum
1. Introduction to MEVs
Maximum Extractable Value (MEV) is a measure representing the total value that can be extracted permissionlessly from the re-ordering, inclusion, or censoring of transactions.
Since 1st January 2020, a staggering $895m of MEV has been extracted within the dark forest of Ethereum, the public Mempool. It is an underground adversarial battle of gas bidding wars between predatory bots attempting to extract profits at the expense of innocent users & network useability.
Without a solution, MEV extraction opportunities would eventually converge towards a monopoly of centralized entities due to economies of scale thus undermining the neutrality and decentralization nature of Ethereum.
Flashbots is a research and development organization funded by Paradigm with the aim of mitigating the negative externalities of MEVs and the uneven playing field. In just about 1 year, it has gained support from more than 90% of the hash rate on Ethereum and is currently the most viable MEV solution in the market.
The goal is to democratize access to MEV revenue, redistribute revenue and increase transparency visibility over MEV activities. It plans to achieve these goals through 3 verticals: Flashbots auctions, Flashbots data, and Flashbots research.
2. Flashbots Auction
Flashbots Auction is an alternative off-chain private communication channel for users to directly communicate their transaction ordering preference to miners via a “bundle” of transactions. It can be thought of as a private secured mempool and a fast-lane transaction.
The key players here are miners, Relayers, and searchers.
Searchers are bots that scan across the blockchain looking for MEV opportunities. Upon finding one, they will create a bundle of transactions with the amount of ETH they are willing to pay (bribe), ordered based on their strategies (sandwich, liquidation, front-run, back-run, Toast, Whitehat), in such a way that maximum profits can be extracted.
There will be multiple searchers, and each of them would send their bundles, privately off-chain, to the Relayers which act as a spam filter, validator, and intermediary between searchers and miners.
Only miners that run a special node off-chain called MEV-Geth are able to receive bundle requests from Relayers and earn extra ETH fees from searchers. (Over 90% of miners on Ethereum have already adopted Flashbots and miners’ overall income has increased by more than $300m)
Due to the limited capacity of block space to fulfill all bundle requests, miners auction their slots through Flashbots auction. All these are done off-chain and searchers with the highest ETH bid specified in the bundles get their transaction included in the block.
Currently, about 54% of all blocks include transaction bundles from Flashbots. Searchers with failed bids would keep retrying for as long as the opportunities still exist, and all these are done off-chain freeing up block capacity for everyone.
In Oct 2021, Flashbots also recently released their public beta Flashbots protection for everyday users who want similar privacy features in fear of being front-run by sandwich bots.
The complexities of submitting bundles to Flashbots auctions are abstracted away to something as simple as adding a new custom RPC URL link in your Metamask while making trades.
Projects like Alchemist have also gone on further to develop a DEX (mistX) that is built on top of Flashbots protect and runs on the Flashbots network. It enables users to send transactions as bundles privately enjoying all the benefits mentioned above.
In this way, Flashbots Auctions have set up an environment where arbitragers, bots & users can broadcast transactions privately without worrying about getting front-run.
The network can be free from endless aggressive Priority Gas Auctions (PGA), and it can be seen from the above chart that the average % of Gas Used for MEV extraction transactions has fallen sharply to less than 1% since Flashbots's inception.
While the plague of MEV in Ethereum has been alleviated to a large extent, more challenges are ahead for Flashbots, and we need more data & research for this to be a success in the long run.
3. Flashbots Data & Flashbots Research
The 2nd vertical is Flashbots data (MEV-Explore), which aims to illuminate the dark forest through a blockchain crawler (MEV-Inspect) that scrapes through Ethereum to identify MEV extraction activities.
The results are translated into real-time intuitive dashboards for greater visibility and transparency to help reduce information asymmetry in this space. In Peter Drucker's words, what doesn't get measured doesn't improve.
MEV-Explorer focuses on quantifying the lower bound approach of MEV extracted on-chain by limiting the scope of search within 8 key DeFi protocols and types of MEV opportunities.
This means the actual extracted MEV out there is way higher than what is estimated on the explorer dashboards. But as Flashbots expand coverage to add more protocols and MEV strategies (CEX-DEX arbitrage, multi-tx/cross-chain arbitrage trades, liquidations, sandwich attacks etc.), it will gradually move towards a more truthful representative state of MEVs.
And finally, the last one is a research hub to further understand the nuances & complexity of MEVs against the backdrop of a dynamic evolving Ethereum network and stakeholders. Insights & new discoveries provided by the Flashbots research arm work hand-in-hand with engineering (Flashbots, auction & data explorers) to increase optimization and efficiency through new tooling, features, models, visualizations, and vice versa.
The key research areas revolve around auction architecture design, in particular, pre-trade privacy, failed trade privacy, efficiency, bundle merging, finality protection, complete privacy, and permissionless. The current Flashbots release is at v0.4, and decentralization/privacy/trust still remains a work-in-progress coordinated effort.
4. MEV Privacy Solutions
While Flashbots have been successful in reducing MEV negative externalities, it is still not completely trustless and private. The current Flashbots Alpha version does not provide privacy guarantees to the searchers meaning that while searchers do not know the bundles of other searchers, miners can still inspect bundles from searchers and front-run them.
It is a system where users and searchers have to trust relayers and miners not to front-run them and miners have to trust the relayers not to censor, and DoS attack them.
Some possible privacy proposals in discussions are for searchers to send only the block headers to miners withholding the transaction bundle details. Miners use PoW to validate the block header, and when a solution is found, transactions are confirmed and included in the block.
The problem is while it solves privacy, the bundle is unknown, and a malicious searcher could send invalid blocks to the miner and DOS the network. Cryptoeconomics could be integrated where searchers have to bond stakes and get slashed upon misbehavior, but the drawback is an increased barrier to entry for new users.
Other methods include trusted hardware solutions (Intel’s SGX secure enclave), timelock encryption, and threshold encryption.
MEV-SGX allows a way for searchers to send their bundles to miners directly without the need for centralized relayers but yet keep their transactions private until mined. It requires the searchers and miners to opt-in with SGX for this to work.
Block construction happens on the searcher end, alongside with truncated header and the Miner SGX’s public key and the miner SGX would provide a PoW solution to decrypt and propagate the block for inclusion.
Timelock encryption helps to temporarily hide the transaction details in the mempool for a period of time longer than usual. The encrypted message can only be decrypted by miners after a certain period of time has passed. The drawback of this is high execution delay and a bad user experience.
Threshold encryption is where each miner owns a share of a decryption key, and a certain threshold is needed to decrypt transactions. The drawback of Distributed Key Generation (DKG) protocol is the trust and reliance that the selected committee will not collude for malicious intentions and that it could be bandwidth expensive.
An example of it is the shutter network where a user sends a transaction to the dark forest, and a group of special nodes known as keepers will only decrypt and execute the transaction after it has left the dark forest and is included in a block.
Each has its trade-off between MEV security, execution delay, and consensus speed time but privacy is yet an important feature to enable a trustless environment of coordination.
5. Flashbots and MEV in ETH 2.0
Even though Flashbots have gained good traction and the adoption rate is over 90% now on Ethereum, the progress would be reset back to 0 again as Ethereum moves from PoW to PoS in ETH 2.0.
In a new environment with new actors (validator pools, exchanges, smaller pool stakers etc.) and a new consensus/execution model (beacon chain merge), it certainly introduces new political dynamics and models for MEV solutions. With that in mind, Flashbots are coming up with a new Eth2 Working Group to collaborate with stakeholders in transitioning towards Ethereum post-merge.
The current proposed architecture for ETH 2.0 is MEV-Boost, and the key difference is the outsourcing of block construction to a distributed network of 3rd party builders otherwise known as the “Proposer Builder Separation” design termed by Vitalik. The rationale is to separate out the selection of block contents from block validation to reduce centralization pressures.
The key players are searchers, builders, Relayers, escrow, and validators.
Searchers send the bundle to the builder who will construct the execution payload (a list of transactions and metadata) together with the block header parameters provided by validators.
Relays receive execution payload from the builder and verify the validity of it, send full execution payload to escrow to provide data availability, and send execution payload headers to validators.
In this way, validators only receive the header, and the visibility of the transaction content is obscured eliminating front-running intentions. The most profitable header is selected from relayers, signed, and returned back to the relayer, and escrow for the block to be propagated to the network.
This way, users and searchers can trust the miners completely so they won't get front-run. Miners do not have to be whitelisted to participate thus helping decentralization.
6. Future of MEVs and Flashbots
While Flashbots have done a great job at democratizing access and creating transparency, redistribution of MEV extraction profits still benefits traders and miners disproportionately rather than the regular users.
Front-running as a service such as KeeperDAO is aiming to change that as MEV profits made from keeper bots are deposited into the ROOK treasury and distributed back to users in the form of ROOK tokens. Same for Eden (ArcherDAO), OpenMEV, and BloXroute with their own profit distribution model that allows users to get a cut.
Within MEVs there are 2 different schools of thought. One is that MEV is unavoidable and inherent in blockchain protocols, we mitigate it (Flashbots), and the other is that it should not even be existent.
Proponents of the latter include Arbitrum & Chainlink opting for a fair sequencing design (FSS). FSS uses decentralized oracle networks to collect user transactions off-chain and generate decentralized consensus for transaction ordering to be submitted to the network. Other applications that remove the MEV problem through different settlement privacy logic (off-chain, batches, encryption, etc.) include CowSwap, Matcha, Secret Swap and etc.
Other general considerations include MEV taxonomy such as formalizing & quantifying it in a more robust, general framework that all stakeholders agreed upon. Some suggestions are the inclusion of the cost of gas fees to search for MEV opportunities that would accrue to miners, regardless of whether the search is successful, but this is not yet considered in the equation.
The current MEV definition both denotes the upper bound extractable and the extracted value taken. Flashbots Research is attempting to refine the definitions and break down the measurement into good/bad MEV extraction or group MEV transactions by actors or would it be better to name Realized Extractable Value (REV) instead of MEV where MEV is the upper bound of REV.
Another development that is gaining fast adoption is L2 rollups on Ethereum. In L2 computation is delegated off-chain to block producers known as sequencers.
Optimism's approach is to act as the auctioneer and auction off the right to sequence and reorder transactions. Optimism would be running the sequencer, and hence centralization risk still exists in the initial phases. Arbitrum fair sequencing model described above is another alternative that is less centralized, but MEV extraction happens at the pre-consensus layer.
In conclusion, Flashbots has undeniably played a successful role in solving the MEV crisis, but as Ethereum moves toward the next milestone, new problems would arise that call for new solutions to ensure the robustness and effectiveness of its goals. The success of Flashbots is highly dependent on the collective efforts of all stakeholders to work together and design a sound system so that the Ethereum network can achieve its vision in the long run.
References:
https://docs.flashbots.net/
https://medium.com/flashbots
https://github.com/flashbots/mev-research
https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest/
https://medium.com/flashbots/frontrunning-the-mev-crisis-40629a613752
https://hackmd.io/@flashbots/quantifying-REV