Mainnet issue prompts IOTA to refine network’s response protocols

The root cause of IOTA’S mainnet incident that took place last week was identified to be related to the infrastructure and deployed software which was unable to process all the transactions into the network.

In a Github report published by Jakub Cech, the Director of Engineering at the IOTA foundation, the incident was addressed as well as the steps taken towards its resolution.

Reportedly, no individual change in the node software, or any other components of the network, led to this event.

Instead, it was triggered by an increased load on the network during a community spam event, during which the network reached over 1000 TPS.

The incident in question took place between November 12, 2020, at 19:45 UTC and November 13, 2020 at 12:37 UTC, by which time it was fully resolved.

A fix for the situation required an update to the node software which led to a new version of Hornet being released and the infrastructure upgraded to the new version of the node software.

The mainnet issue did, however, help the development team get a clearer grasp of certain existing vulnerabilities such as the IOPS limits on the infrastructure around the coordinator, the need for a redundancy mechanism to ensure the retention of transactions.

It was also found that the coordinator needs to react to the network state and pause milestone issuance automatically in situations that approach the limit of hardware resources.

The development team also stated,

“This incident and the preceding spam event have helped us to refine our response protocols. We would like to thank our community for their patience. A huge thank you also goes to our engineering team, and the Hornet team, who worked late into the night until a fix was devised and ready to be tested.”