16 March 2023 public testnet stress-tests

Why

One of the main point features of TON is the architecture built to process millions of transactions simultaneously through infinite horizontal sharding which, being envisioned and described in 2017, is still one of the most advanced blockchain projects.

However, while theoretically possible, on the way to practical implementation several challenges still need to be solved. These challenges include stability and security of the network when each node validates only part of the network, thorough alignment of incentives of all actors, a balance between low fees and spam attack surface, and building trustless and decentralized but achievable infrastructure.

On the one hand, the solution to these challenges must go hand in hand with the needs, so as not to overcomplicate the work with TON for no reason. On the other hand, network load is intermittent, and often one popular app, game, or lucrative protocol may multiply the number of transactions in seconds. Thus for TON to provide a smooth and limitless experience to users, it is necessary to be ready for throughput much higher than average.

That is why despite the ongoing advancement of the implementation of the infinite horizontal sharding paradigm, we at TF regularly measure performance and optimize node to current loads.

How

For proper performance measurements which give information about real network behavior, it is important to study the profile of the current load. Currently, key TON usage is related to a) simple TON transfers, b) NFT transfers, and c) Jetton (token) transfers. The main load can be attributed to the last one since jetton logic is relatively heavy (each of both jetton wallets participating in the transfer need to authorize each other) and many transfers are part of relatively long transaction chains of DEX swaps. For now (data obtained for mainnet February 2023) average transaction chain length is >3.5, that is there are about 4 transactions per user interaction. The average load in TON is about 2 TPS with accidental spikes up to 7-10 TPS during new product launching (such as megaton.fi farming program or lost dogs NFT collection sale).

It is also important to note that while TON is a multi-blockchain system, currently the main activity happens in workchain 0, with the only exception of TON governance-related activity in workchain -1. While it is relatively easy to multiply throughput by spawning new isolated workchains, usually it is not an optimal way of scaling since it hinders the network effect of any user easily interacting with any other user or DApp. That is why, for now, we focus on the throughput performance of workchains themselves.

For modeling such loads we have developed a jetton-like stress-test contract system. Depending on submitted parameters it starts an arbitrarily long chain of transactions between stress tester contract instances. These transactions include all features specific to a real load: signature+seqno-based authentication of the external message, deployment counterparty on demand, and address-based authentication characteristic for jettons. Parameters include the length of the chains, number of chain forks, preference (probability of the next destination being in the same shard), and additional payload. For simplicity, further, we would call the stress-test contract instances jetton wallets since they are equivalent to real jettons.

Performance measurements were carried out in a public testnet on 16 March 2023, the number of shards was limited by 16. At the time of the test, there were 12 validators in testnet spreaded across the globe: Helsinki Finland, Berlin Germany, Khmelnitsky Ukraine, New York, Dallas and Los Angeles USA. Validators had 32 to 68 GB RAM (6 - 32 GB, 1 - 64 GB, 5 - 68 GB) and 8-24 CPU (2 - 8 CPU, 8 - 12 CPU, 1 - 16 CPU, 1 - 24 CPU).
Results can be verified via public API/explorers https://testnet.toncenter.com/, https://testnet.dton.io/, https://testnet.tonapi.io/ and https://test-explorer.toncoin.org/

Results

We have carried out a number of tests in which we described their relevance to real network performance.

Isotropic jetton load

As part of this test, external messages were sent to the network inducing jetton-jetton chains of transactions with a length of 4, and all addresses were evenly distributed over the workchain. Such loads were tested for instance from 13:00 UTC to 13:20 UTC

This test demonstrates average 700 TPS (transactions per second) throughput.

Isotropic ton transfers load

Load related to toncoin transfers is more liteweight: for interaction in which each external message induced 4 toncoin transfer transactions around 900 TPS can be achieved.

Shard optimized load

When destination of the messages lay in the same shard as the source, cross-shard activity is minimized, thus there are no overhead related to shard synchronization and much higher TPS are achievable. In particular we have managed to demonstrate 2k TPS (see below) in public testnet. However, It should be emphasized that usage of “Large” smart contracts, which can have separate instances in different shardchains and which can fully utilize that performance trick is not yet fully supported by the network.

Conclusions

Several conclusions can be drawn from obtained results:

• Current practical throughput is at least 100 times higher than the average load of TON mainnet, which means that TON ecosystem is ready for massive userbase growth as well as a sudden burst of load at the time of the launch of popular DApps
• TON has time to thoroughly and carefully test the transition to the next big step in horizontal scaling: validator/collator separation
• Comparison with results obtained in Solana and Everscale networks shows that TON has room to further optimization of throughput even with the current approach.