Nov 18, 23:13 UTC
The engineering team has successfully completed the maintenance on IMF nodes in order to stabilize services. All the services have returned back to operational status. We will provide a detailed post mortem once a full investigation is completed. Services were fully restored at 20:00 UTC, monitoring continued to ensure stability.
Incident Start Time: 11-16-2021 11:14:00
Incident End Time:11-16-2021 20:00:00
Total Duration:8 hours and 46 minutes
Nov 17, 01:34 UTC
The engineering team has performed multiple maintenance operations to further stabilize the environment.
Monitoring will continue.
Further updates to come.
Nov 16, 23:21 UTC
Services are stabilized but we are continuing to monitor to ensure complete functionality.
We will provide updates as we received them.
Nov 16, 21:15 UTC
A fix has been implemented by our engineering team and emails are now being received in a timely manner. We will monitor the results.
Nov 16, 20:33 UTC
Engineering continue to investigate the cause of high load, backend processes are starting to stabilize though there is still some noticeable delay. We will provide updates based on progress.
Nov 16, 18:59 UTC
The engineering teams are adding resources to alleviate high load in multiple backend services, we are continuing to work to bring services back to fully functioning. We're engaging additional teams to confirm there are no network issues.
Nov 16, 16:48 UTC
Engineering operations team continues to work to bring service back up, we will continue to provide updates based on progress. Inbound delays are still expected.
Nov 16, 16:00 UTC
Engineers have identified issues with login once again, we're continuing to investigate the cause and are working to restore services. Email delays are expected to continue for the time being.
Nov 16, 15:14 UTC
Our engineers are continuing work to bring services back online, some impacted users may start to see mail flow through and login success. We will provide updates based on progress.
Nov 16, 14:29 UTC
Our team is still currently working on this issue. Customers still may see some delays with inbound emails.
Nov 16, 13:45 UTC
We are aware of an issue with inbound and outbound emails on Cluster A.
Users experience the following error message "4.7.1 Service unavailable - try again later"
Our engineering team has been engaged and is investigating this issue at this time.
Nov 16, 11:25 UTC