Webmail/IMAP/POP/SMTP - Cluster B
Incident Report for OpenSRS
Postmortem

Incident Date: April 4, 2020
Incident Number: PR-1031

On April 4, 2020, at 3:14 PM ET, Tucows HostedEmail and Domains platform experienced service interruption and email delivery issues in prod B. Operations team was engaged to investigate the issue.

The root cause was determined to be a service failure that caused multiple IMAP nodes to appear offline.

At 5:20 PM ET, The Operations team restored all the services by completing a rolling restart on all of the impacted IMAP nodes.

Tucows’ Operations team completed emergency maintenance to automatically restart the service upon failure to improve the recovery time.

Tucows is to set up additional monitoring to observe and address such failures in a timely manner.

Thank you,

Tucows Operations

Posted Apr 08, 2020 - 19:16 UTC

Resolved
This incident has been resolved.
Posted Apr 04, 2020 - 20:18 UTC
Investigating
Users will experience issues with authenticating themselves for the hostedemail service on cluster B.

As a result, they may not be able to pull any new mail, send new mail, or access webmail altogether.

Next update: 20:15UTC
Posted Apr 04, 2020 - 19:32 UTC
This incident affected: Hosted Email (Cluster B, Webmail).