Incident Date: November 19, 2021
Incident Number: PR-2593
On November 19, 2021 at 7:55 PM ET, Tucows’ hosted email platform experienced service interruption impacting Webmail in Prod B. Tucows’ Network Engineering team was engaged to investigate the issue.
The service interruption was caused due to a DDoS attack. The abusive traffic was mitigated however it caused unexpected behaviour with one of our load balancers.
Tucows’ Engineering team increased the severity of the incident when we observed the external impact.
At 9:55 PM ET, The engineering team failover the services to the secondary load balancer and restarted webmail nodes in a controlled manner to restore all the services successfully.
Tucows is to continue working with existing vendors to improve DDoS mitigation services to address and rectify DDoS attacks in a timely manner.
Tucows is to work with the vendor to investigate the root cause of the unexpected behavior on the load balancer.
Tucows is to enhance the monitoring for better visibility.
Thank you,
Tucows Engineering Team