July 21, 2021
Two weeks ago the Weatherwax testnet was launched and is currently at height 8120. The Android and iOS apps are still on the Stibbons network but will be updated soon.
To test out the new network, which enables additional functionality via TariScript, a stress test was performed. Six console wallets each sent 6000 transactions to 10 receiving console wallets, totalling 36000 transactions.
We were looking for a number of things during the stress test.
- Were all transactions mined?
- Did the wallets end up with the correct status?
- Were there any chain splits or reorgs that could not be recovered?
- Did the transactions reach all the miners?
The main aim of the stress test is to find bugs and inefficiencies that can be fixed, and to determine reasonable settings for rate limiting and banning nodes.
The results were very good. The main finding was that many nodes banned each other due to message flooding. This has a knock on effect on transaction propagation and reorgs. When nodes get banned, small chain forks occur and when they are unbanned, the strongest chain wins. The largest reorg was 17 blocks, and all nodes handled this without any errors. This is a great result.
The stats were:
|Sender wallet||wallet 01||wallet 02||wallet 03||wallet 04||wallet 05||wallet 06||Totals or Avg|
|Submit time (min.)||101||76||70||88||88||79||84|
|Highest single txn send count||79||54||54||68||71||53||379|
|End status: Broadcast||21||0||4||23||0||0||48|
|End status: MinedUnconfirmed||0||6||3||0||1||0||10|
|End status: MinedConfirmed||5979||5994||5993||5977||5999||6000||35942|
|MinedConfirmed & Valid||5972||5994||5990||5975||5999||6000||35930|
|Blocks to mine all||420||472||472||481||418||473||456|
|Mined success ratio||99.533%||99.900%||99.833%||99.583%||99.983%||100.000%||99.806%|
The first stress test block (#5681) was mined Thu, 15 Jul 2021 16:16:35 +0000, and the last stress test block (#6162) was mined Fri, 16 Jul 2021 08:27:59 +0000, thus lasting just under 16.5 hours.
Were all transactions mined?
Out of the 36000 transactions, 48 were still in the broadcast stage, meaning that they were sent to a base node, but for some reason did not make it into the chain. This is most likely to a reorg, and the wallet should probably retry these.
Did the wallets end up with the correct status
35,930 out of 36,000 transactions were mined within 16.5hrs, translating to a success rate of 99.806%. This makes this the most successful stress test of the Tari network to date. However, we’re still chasing that elusive 100% goal.
Were there any chain splits that could not be recovered
There were none. We were very pleased to see the nodes reorg to stronger chains and continue processing blocks.
Did the transactions reach all the miners
Although the vast majority of transactions were mined, there were times when some nodes had full mempools, but empty or lean blocks were being added to the chain. This means that the miners mining them were not aware of the transactions in the mempool. We believe this is mainly due to the banning issue. We are actively investigating this and will be the focus of our next stress test.
Note: The final batch of transactions was mined from block #6081 onwards by manually connecting a miner to base nodes that still had unmined transactions in their mempools. Transactions that were mined with this manual intervention totalled 11,764 or 32.7%.
The way forward
The next stress test will seek to mitigate the banning issue and improve consistency in mempool populations across the network.
After that will be to repeat the test using one-sided payments (non-interactive transactions using TariScript). This is exciting since it will be the first test of TariScript in “real world” conditions.