The Race Against Time and Strategy: About the AnySwap Rescue and Things We Have Learnt
By BlockSec Team
On Jan 18th, our monitoring system detected an attack against the AnySwap project (aka Multichain). The vulnerability is due to the flawed
anySwapOutUnderlyingWithPermit() function whose verification mechanism can be bypassed to withdraw the approved tokens.
Although the project has adopted different approaches (e.g., sending transactions to the users, as shown in Figure 1) to notify affected users, a number of them failed to take timely actions (i.e., revoking the approvals) to protect their funds. As a result, attackers could continuously launch the attack to get victim’s funds.
To protect potential victims, after much deliberation, our team decided to perform an emergency rescue for the AnySwap vulnerable contract (0x6b7a87899490EcE95443e979cA9485CBE7E71522) on Ethereum. Specifically, we can transfer the vulnerable account’s funds to our white hat wallet, which is a mulsig-wallet (0xd186540FbCc460f6a3A9e705DC6d2406cBcc1C47) on Ethereum.
To make our white-hat rescue transparent, we documented our intention in a pdf file and shared the file hash with the community. This can differentiate our emergency rescue from the attack, while at the same time do not leak the details (since the vulnerability can still be exploited.) We launched the emergency rescue from Jan 21st, 2022 to Mar 11th, 2022, and the corresponding messages had been released to the public, as follows:
The emergency rescue is not a trivial task. Indeed, there exist several challenges, including both technical and non-technical issues, to make a successful rescue. As the emergency rescue already terminated, we can comprehensively review the whole process and share the lessons we have learnt. We believe this experience can shed some lights on securing the DeFi ecosystem.
Key takeaways (TL;DR)
- There exist competitions between different participants, including the whitehats and the attackers. The payment fee for the Flashbots increased rapidly as times went by.
- Flashbots did not always work. Instead, some attackers turned to use the normal mempool to launch a successful attack by adopting some sophisticated strategies.
- Some attacker was whitewashed by returning back part of the stolen fund, while the remaining was kept as the bounty award. This phenomenon, though not the first appearance, is controversial in the community because such an incentive may not be fair to the real whitehats.
- To convince the community, it is a good practice for the whitehat to make the action public known in advance without leaking any detailed sensitive information.
- The community can work together to perform the resuce more effectively and efficiently. For example, building a coordination mechanism to reduce/avoid whitehat competitions.
In the following, we first give the whole picture of the rescue during this period. Then, we illustrate our way to perform the rescue and the challenges need to be solved. After that, we discuss some lessons we have learnt from the rescue. Finally, we provide some thoughts/suggestions which might be meaningful to secure the ecosystem.
0x1 Rescues and Attacks
0x1.1 Overall result
The attacks and rescues we observed and investigated in this report have been last for months, ranging from block 14028474 (Jan 18th, 2022) to block 14421215 (Mar 20th, 2022).
The accounts that performed rescues and attacks are summarized in the following table. For the sake of simplicity, only the first four bits of the address are given to represent an EOA. An account is either a rescue account or an attack account. It is worth noting that the type of an account is determined based on either the labels marked by Etherscan.io, or the transferring destination addresses we observed.
In total, there are 9 rescue accounts which rescued 483.027693 ETH (with fee 295.970554 ETH, i.e., 61.27% of the total amount) and 21 attack accounts that harvested 1433.092224 ETH (with fee 148.903707 ETH, i.e., 10.39% of the total amount), respectively. It is worth noting that the loss is a roughly estimate due to some complicated interactions. For instance, an attack account might turn into a rescue account after negotiating with AnySwap, and we will discuss such a phenomenon later. The last column of the table indicates the fee sent to the miner, which were used to win the competition of using the Flashbots.
0x1.2 The trend of the fee to bid the Flashbots
As mentioned earlier, whitehats need to compete with attackers to send the transactions. As a result, the percentage of the fee to the miner (in the Flashbots transactions) may reflect the level of the competition. To quantitatively measure it, we investigate the fee percentage (including attack transactions and rescue transactions respectively) for each block.
Figure 4 gives the trend we observed so far (from block 14028474 to block 14369199). The first several attack transactions do not involve any fee, which means few (even no) competitions were there during that period. This is reasonable because these early attacks might not be well known to others.
As a matter of fact, the first attack with fee (10%) occurred at block 14029765. Since then, the fee percentage increased rapidly as more participants were involved. For example, the percentage reached 80% at block 14072385, and soon reached 91% at block 14129449.
In short, the trend suggests that it is definitely an arms race to win the competition by adjusting more fee to the miner(s).
0x2 Our Rescue and the Challenges
0x2.1 The way to perform the rescue
The basic idea to perform the rescue is quite straightforward. Specifically, we need to monitor the accounts that have approved WETH to the vulnerable contract. When there is any WETH transferred to the account, we can directly transfer it to our multi-sig wallet by exploiting the vulnerable AnySwap contract. The key requirements are:
- R1: Efficiently locating the transactions that transfer tokens to the victim accounts. We name these transactions as transferred TXs in the following.
- R2: Correctly crafting the transactions to perform the rescue. We name these transactions as rescue TXs in the following.
- R3: Successfully frontrunning the transactions sent by the attackers (and any other third-parties). We name these transactions as attack TXs in the following.
R1 and R2 are not obstacles for us. Specifically, we have built an internal system to monitor the mempool, which allows us to timely locate the transferring transactions. Meanwhile, we have developed a tool to build the transactions automatically as well.
However, R3 remains a challenge. One may expect that the Flashbots can be used to win the competition. However, it is not that easy to achieve the goal. Firstly, the attackers may also adopt the Flashbots. As a fee-bidding system, the successful rate may depend on the fee specified to the miner. The strategy to set the fee needs to be determined. Secondly, using Flashbots might not be a good choice cause the intensive competition. Therefore, we also send normal transactions using the mempool. To guarantee the succeed, the strategy to place the transaction in the right position needs to be considered. Finally, we also battle with other whitehats, whose behavior seems to be suspicious in some cases.
0x2.2 Competitions we involved
In total, we attempted to protect distinct 171 potential victims, while 10 of them protected themselves by revoking the approvals right before we were trying to perform the rescue. For the remaining 161 valid victims, we only succeeded to rescue 14 of them due to the competitions. The failure cases are summarized in the following table, involving 3 rescue accounts and 16 attack accounts.
Hence there do exist some lessons we are willing to share with the community.
0x3 Some Lessons We have Learnt
0x3.1 How to determine the number of fee to Flashbots’ miner?
In summary, we were beaten by 12 competitors, including 2 rescue accounts and 10 attack accounts, which were using the Flashbots.
Our strategy to set the miner fee was quite conservative. Specifically, we aimed to protect the victims by spending the fee as little as possible. Hence when we did not actively use/increase the fee unless some successful attack TXs had already set the fee. For example, if an attacker sets the fee to 10% of the fund, then we might use 11% for the next rescue to compete with that attacker. However, the result suggests that this strategy did not work as the attackers (or even some whitehats) often (if not always) aggressively increased the fee to beat others, as follows:
- Figure 5 shows that the attacker 0x5738** set the fee percentage to 70% at block 14071986.
- Figure 6 shows that the whitehat 0x14ca** set the fee percentage to 79% at block 14072255.
- Figure 7 shows that the whitehat 0x14ca** set the fee percentage to 80%, at block 14072385.
- Figure 8 shows that the whitehat 0x9117** set the fee percentage to 81% at block 14072417.
- Figure 9 shows that the attacker 0x5738** set the fee percentage to 86% at block 14073395.
In brief, it seems to be a zero-sum game which requires modelling the behaviors of different participants to win the competition.
In practice, however, it is challenging to figure out better/optimal strategies, and meanwhile to reduce the cost as much as possible.
0x3.2 How to place the right position in the mempool?
Now it looks like the rescue would rely on the arms race of fee competition to bid the Flashbots. However, we found that using Flashbots was not panacea due to the intense competitions from other participants which had nothing to do with the rescuing and attacking. In such a case, even the highest fee specified by an attack Tx cannot guarantee to win the chance to use the Flashbots.
Alternatively, a normal transaction with the right position in the mempool may have the opportunity to achieve the goal. The right position here means the rescue/attack TX should be placed at the position behind the transferred TX, and the position should be very close to the transferred TX (the closer, the better). Note that by using the strategy, the attacker 0x48e9** harvested 312.014657 ETH without paying any fee to Flashbots miners.
The following four figures demonstrate the top 2 biggest profits the attacker have made:
- Figure 10 shows that a victim deposited 50 ETH at position 65 of block 14051020, and figure 11 shows that the attacker harvested the 50 ETH at position 66 of the same block.
- Figure 12 shows that a victim deposited 200 ETH at position 161 of block 14052155, and figure 13 shows that the attacker harvested the 200 ETH at position 164 of the same block.
Obviously, this sophisticated strategy is quite useful and enlightening, and deserves more attentions and efforts to learn something from it.
0x4 Some Other Thoughts
0x4.1 Whitehat hack or attack?
When it comes to the recognition of the whitehat hacks, they might be not as straightforward as one may expect.
For example, address 0xfa27 was labelled as Multichain Exploiter 4 (Whitehat) by Etherscan.io. Actually, at the very beginning, it was labelled as Multichain Exploiter 4. After several rounds of negotiation between the attacker and the AnySwap project, the attacker was persuaded to return back part of the stolen funds.
- In TX 0x3c3d**, AnySwap contacted the attacker:
First and foremost thanks for getting the weth. I was not aware of the hack and realized the situation only because the weth never arrived in my wallet after the cowswap transaction. Considering the amount at stake, would you accept 50 eth as a fair tip? Here is mine txn: https://etherscan.io/tx/0x2db9a6a51604e2be8b2c3469773afb201f0b48a318fb7e5f5e49175e818df5ba https://etherscan.io/tx/0xe50ed602bd916fc304d53c4fed236698b71691a95774ff0aeeb74b699c6227f7
- In TX 0xd360**, the attacker replied:
pls verify by sending me back a tx. I will send you the rest 258 eth. 39 eth went to the miners because there other attackers so I have to pay that tip to save the money.
- In TX 0x354f**, AnySwap acknowledged with thanks after receiving the funds:
Well received, thank you for your honesty.
Obviously, this attacker was whitewashed and also made some profits from the attack. The similar cases have occurred several times in the past, and they are still controversial in the community because such an incentive may not be fair.
0x4.2 Competition between whitehat hacks?
It is necessary to build a coordination mechanism to reduce/avoid competitions between the whitehats. Such a competition inevitably leads to a waste of the rescue power. In this rescue, there exist 54 victims (with 450 ETH fund) which were protected by other three whitehats, while we also tried to perform the rescue.
The competition between whitehats not only wasted the rescue power, but also raised the cost to perform the rescue. For example, as shown in figure 7 and figure 8, the fee spent by the two rescue TXs with different whitehats are 80% and 81% respectively.
Unfortunately, the whitehats will not step back unless there exists any mechanism to coordinate with each other. Otherwise, it is impossible to make the competition disappear.
0x4.3 How to make a better rescue
On the one side, to convince the community, it is a good practice for the whitehat to make the action public known in advance without leaking any detailed sensitive information. There is enough time to do it as the rescue is usually a seesaw battle with multiple trials, which is different from some one-time effort like blocking a particular attack. Of course, the detailed information regarding the vulnerabilities should not be leaked.
To achieve this, the detailed information will not be disclosed at the very beginning and will be disclosed to the community after completing the rescue, just as what we have done for the AnySwap rescue. However, the hash of the document with the whitehat intention can be shared with the community.
On the other side, the community can do more things to perform the resuce more effectively and efficiently, including, but not limited to:
- Flashbots/Miner may provide some the green channel for the certified whitehats. The green channel can provide a high priority to front-run the attackers’ TXs, and avoid competition between whitehats.
- The projects being attacked cover the cost of Flashbots/miners.
- The projects may apply convenient and fast notification mechanisms to users.
- The project may apply the emergency mechanism in the code.