Measuring Exchange Quality And ‘Fake Volume’

By: Victor Ramirez

Key Takeaways: 

• Centralized exchanges can vary widely in quality based on transparency, resilience, data quality, and other factors.

• Trade patterns, volume correlations and price discovery characteristics can test authenticity and health of exchange activity.

Introduction

Centralized exchanges (CEX’s) serve as a crucial intermediary between crypto market participants and blockchain networks, facilitating billions of dollars of value transferred each day. Many user’s first interaction with crypto is likely to occur through an exchange. However, exchanges vary greatly in quality across their primary functions. In this issue of Coin Metrics’ State of the Network, we’ll delve into how we assess exchange quality. 

Trusted Exchange Framework: An Overview

As the use-case for crypto data has expanded, the importance of precise, high quality data has been further emphasized. Coin Metrics created the Trusted Exchange Framework to vet which exchanges should be included to calculate precise market metrics as well as select the highest quality constituents for indexes. We assess exchanges based on the following categories:

• Data Quality: The level of confidence that the exchange’s reported data is accurate.

• Transparency: The quality of publicly disclosed information from an exchange, such as its quality of its Proof of Reserves or its publicly disclosed balance sheets.

• Resilience & Security: How well an exchange protects its users against market and security risks.

• Regulatory Compliance: An exchange’s ability to meet regulatory requirements via its existing licenses, adjusted by the relative of the regulatory environment it conducts business.

• API Quality: An exchange’s quality to be used as a programmable entity.

Based on this criteria, we arrive at the following exchange rankings below.

Source: Coin Metrics Trusted Exchange Framework

Details of the methodology can be found here.

Trading Volume Among Trusted Exchanges

During the early stages of the crypto industry, it was common for exchanges to artificially inflate their trading volume in order to attract customers and falsely demonstrate legitimacy. In 2019, Bitwise conducted a comprehensive analysis of fake volume in crypto exchanges in which they claimed that 95% of volume is “fake”. Inspired by Bitwise’s methodology, we created a series of tests for detecting and filtering out fake volume. Thus we can arrive at our best-guess estimate for crypto trading volume, which we call our “Trusted Volume”.

Source: Coin Metrics Market Data Feed

As of April 26^th, 2024, trusted volume on a rolling average weekly window is at $40 billion, about half of the $80 billion peak this year and a little over a third from the all-time highs of May 2021. For comparison, NYSE and NASDAQ daily trading volumes hover on the order of $1-2 trillion according to CBOE data. Note that this trusted volume metric is a lower bound estimate; DEXes and centralized exchanges outside of our coverage universe are not included. 

This raises the question of what percentage of all crypto volume is fake. It depends on how you count all fake volume. Any exchange can theoretically generate an infinite amount of fake volume from wash-trading bots. Additionally, fake volume tends to not be evenly distributed across exchanges: regulated, reputable exchanges have a significantly lower proportion of its total volume as fake, while less reputable exchanges tend to have a higher or even majority of its total volume to be fake. In reality, the vast majority of real usage happens on exchanges where most of its activity appears organic. Factoring these altogether to calculate a percentage of fake volume could thus lead to misleading conclusions about the legitimacy of crypto market activity.

How are we able to filter out fake volume? By collecting data directly from exchanges, we can apply fake-volume tests developed by the industry to flag unusual trading activity.

Testing for Wash Trading

Buy/Sell Trade Patterns

The pattern of buys and sells for an exchange can reveal the presence of highly unusual market activity. Legitimate market activity tends to heavily skew towards several consecutive buy or sell trades due to the presence of informed traders that are willing to cross the spread and take liquidity in response to material new information. On the other hand, exchanges that have historically fabricated volume have an even distribution of buys and sells, resembling that of a random coin-toss, as shown by a report from DAR. This is likely due to wash trading, non-economic trading activity, or other trading activity generated from an artificial process. 

The chart below illustrates the difference between well-behaved markets such as binance-btc-usdt and coinbase-btc-usd versus markets that display unusual trading activity. The x-axis represents the buy-sell sequences from trades data (b = buy, s = sell) and the y-axis represents the frequency.

Source: Coin Metrics Market Data Feed 

Volume Correlation

Another way to detect fake volume is to observe the correlation of the relative change in trading activity volume across exchanges. Exchanges that exhibited fake volume tended to have volume fluctuations which did not correlate to the rest of the exchange volumes. In general, well-behaved exchanges behave the same way—by following the market—while ill-behaved exchanges misbehave in different ways.

We can illustrate this by looking at the volume reported from a sample of exchanges the week that the Bitcoin Spot ETF started trading in the US. US business hours are shaded in gray while APAC business hours are shaded in red.

Source: Coin Metrics Market Data Feed 

Note that there are some general seasonality patterns: volumes tend to spike at the open of US business hours, stay at some elevated level until close of business, then fall off during the weekend (1/21 to 1/22). However, fluctuations are expected due to unforeseen market events.

In the reported volume charts above, we see Coinbase and Binance follow the same volume pattern, noting that Binance follows this general seasonality despite not being available in the US. Importantly, the “amplitude” of the trading volume resets per day. Poloniex does not follow that same intraday seasonality and in general shows relatively small fluctuations throughout the day. Upbit, a Singaporean exchange that’s one of the largest Asian exchanges by reported volume, has a trading volume that is also unusual but for a different reason: its spikes in trading volume appear to not correlate with the major exchanges, and there appears to be no dip in trading activity during the weekend. 

These patterns can also be detected when looking at the percent change in volume over time.

Source: Coin Metrics Market Data Feed

Benford’s Law

Benford’s Law states that leading digits tend to be low and most frequent, and the frequencies decrease as the leading digit increases. Benford’s Law is commonly used to detect fraud in financial (such as trade amounts in traditional markets) and non-financial applications (such as vote counts in elections). We can apply Benford’s Law to trades data from crypto exchanges by tallying the leading digits of the trade value. If the distribution of leading digits violates this pattern, it's a signal of manipulated behavior.

An example of a market following (binance-btc-usdt-spot) and violating (bibox-btc-usdt-spot) Benford’s Law is shown below. Generally speaking, most of the markets we studied (BTC-Stables, ETH-Stables, BTC/ETH-USD) tend to fit Benford’s Law.

Source: Coin Metrics Market Data Feed 

Price Discovery Among Exchanges: Where Prices Lead and Lag

Price discovery is the process by which the market agrees on a price. In an efficient market, we should expect prices to be tightly coupled across market makers. Due to the variability in regulatory compliance, liquidity, and overall market quality across crypto exchanges, we see asymmetries in prices reported by these exchanges, leading to potential arbitrage opportunities.

Price discovery across exchanges can be measured with a lead-lag analysis using the Hayashi-Yoshida estimator that Robertson and Zhang applied to Bitcoin spot markets in 2022. In this method, returns of an asset on one market are shifted forwards and backwards in time relative to the returns of the same asset on a reference market. Estimating the correlation between these returns as a function of time displacement lets you observe how much a given market should be lagged for its prices to most strongly correlate with the adjacent reference market. 

Consider the lag curves for the example markets below. Coinbase and Gemini’s BTC-USD markets most correlate at 0 seconds, meaning these markets are in sync. A counter-example is between Binance and Upbit, where Upbit’s maximum correlation to Binance’s prices occur at –13.9 seconds. This means that Upbit’s BTC-USDT market price lags behind Binance’s prices by an average of 13.9 seconds.

Source: Coin Metrics Trusted Exchange Framework

You can observe general price instability by looking at the price charts on a 1-minute frequency. Below is a comparison of 1-minute prices for exchanges that are typically used for reference rates compared to Upbit.

Conclusion

The auditability of data in crypto is one of the industry’s most defining characteristics. By looking through patterns of openly available exchange data, we can detect signs of market irregularities and create a holistic assessment on the quality of exchanges. Although the crypto market still shows signs of immaturity, the transparent nature of how crypto operates helps identify sources of market inefficiency and ultimately motivate improvements in vital market infrastructure. 

Coin Metrics Updates

This week’s updates from the Coin Metrics team:

• Follow Coin Metrics’ State of the Market newsletter which contextualizes the week’s crypto market movements with concise commentary, rich visuals, and timely data.

Subscribe and Past Issues

As always, if you have any feedback or requests please let us know here.

Coin Metrics’ State of the Network, is an unbiased, weekly view of the crypto market informed by our own network (on-chain) and market data.

If you'd like to get State of the Network in your inbox, please subscribe here. You can see previous issues of State of the Network here.

© 2024 Coin Metrics Inc. All rights reserved. Redistribution is not permitted without consent. This newsletter does not constitute investment advice and is for informational purposes only and you should not make an investment decision on the basis of this information. The newsletter is provided “as is” and Coin Metrics will not be liable for any loss or damage resulting from information obtained from the newsletter.