2022–01–11 Incident Report
This article has been updated for the 2022–01–17 incident when raydium api.raydium.io was returning abnormal lp_price. Due to recent instability related to the pricing report of Raydium LP pools, we will pause on adding new Raydium LP pools
Impacted users may refer to the additional list of wallet addresses below and reach out to us on Discord for any additional questions. All funds are safe, and we have already pursued the same compensation scheme for all wallets that experienced greater than $1 in liquidation value.
What took place
At around 1530pm 11th Jan 2022 (UTC), we upgraded our pricing mechanism for SOL-USDC and SOL-USDT Raydium LP. This upgraded pricing mechanism deviated from Raydium’s official API pricing data as it contained a synchronization issue.
While the upgraded pricing mechanism behaved normally most of the time, it also occasionally led to unstable prices being recorded in Apricot. This issue existed for the SOL-USDC, SOL-USDT, and RAY-USDC Raydium LP. Prices for Orca LPs and all other tokens were not affected.
It was identified and fixed at around 0230am 12th Jan 2022 (UTC). The prices of affected LP tokens are back to normal and stable since then.
How users were affected
This instability in the LP token prices affected our users who had significant positions in the SOL-USDC, SOL-USDT, and RAY-USDC Raydium LPs. It affected these users in two ways:
- Some users had their LP tokens forcibly redeemed by Apricot. This happened because Apricot’s liquidation engine, using incorrect LP prices, reached the conclusion that these users were near liquidation, and as part of the liquidation procedure forcibly redeemed some of these users’ LP tokens. For this group of users, this forced redemption actually returned their account status to normal, and stopped actual liquidation (which carries a 4% liquidation penalty) from taking place.
- Some users had their positions partially liquidated. This resulted in some of their collateral being sold and used to repay their debt. As a result, for this group of users there is a direct 4% liquidation penalty incurred on them.
Compensation for affected users
For users who were erroneously liquidated and who experienced a 4% liquidation penalty as a consequence, we will return 5% of the liquidation value in USDC directly to the affected wallets. The 25% increase is an expression of our sincere apology.
Transfer to the liquidated wallets will be done within 24 hrs (only listing wallets that experienced > $1 liquidation value):
BqfLWMq…MssfetW 4CmdnnA…of4CyEQ 6VGcEyf…GjXefPN E7FbMzq…GqHPX8v
Fu8tW3K…CSHqTzo 9EuPejL…9yHzZMN BFAEoGb…w1zuFX8 5w81nKC…qXDD7bh
HDFigdD…pWe1PBi GoSAGHm…oRhEdn6 8oDBAVj…Cuy7Nib 2RXQv4v…Dcp6n9w
4d4HJoH…j9SjVpN 8t2r2pa…iNDnCfA 12DMsVw…RecF43M ESSFrGB…9EYd1DW
Ebwnz3i…o4PA9tb EmEia62…27YME3X Br6E8VR…cq6eAKe 9QiFNdb…d7zqdt1
For users whose LP tokens were forcibly redeemed but who were otherwise not liquidated, as it strictly did not affect their Net Deposit, we do not have additional compensation planned.
Last, but not least
Again, we sincerely apologize for the confusion and inconvenience caused by this incident. We will use this as a lesson to further strengthen our testing requirement and risk management practice. We look forward to your continued support as we continue to improve all aspects of Apricot. 🍑