Updates related to Incident 8694

Domain Service is Online

Updated Tuesday, October 13th, 2009 at 3:24 PM ET
2009-10-13 at 19:24 UTC - Other time zones

OpenSRS Domains services via the Reseller Web Interface (RWI) and Application protocol (API) are fully available. We tested services and confirm that the intermittent timeout/delay issue is resolved.

Update 20:57 UTC/16:57 ET:
We continue our investigation and will provide an incident summary by end of business day October 14, 2009.

Update October 14, 2009 17:07 ET/21:07 UTC
An intermittent delay/timeout issue occurred for customers attempting to do look-ups and renewals. Resellers using the Reseller Web Interface (RWI) began reporting the intermittent issue between 13:20 and 14:00 ET. Our Support team began investigating. We were implementing a regular code change. This was fully reverted by 14:29 ET and services via RWI were restored. At 14:41 ET our network monitoring system received alarms for the API. We advised customers of the degraded services (intermittent delays and timeouts). At 15:13 ET, we investigated and confirmed that the issue was resolved. The cause of the alarms for API servers was a result of high load while we reverted the code change. We continued to closely monitor for any residual impact. Service was fully restored by 15:34 ET.

This update is related to

Storefront is Online

Updated Tuesday, October 13th, 2009 at 3:22 PM ET
2009-10-13 at 19:22 UTC - Other time zones

Storefront Resellers and their customers can now access all Storefront services. We tested services and confirm that the intermittent timeout/delay issue is resolved.

Update 20:57 UTC/16:57 ET:
We continue our investigation and will provide an incident summary by end of business day October 14, 2009.

Update October 14, 2009 21:24 UTC/17:15 ET:
An intermittent delay/timeout issue occurred for customers attempting to do look-ups and renewals. Storefront Resellers and their customers may have experienced these issues between 13:20 and 14:00 ET. Our Support team began investigating. We were implementing a regular code change. This was fully reverted by 14:29 ET and services were restored. We continued to closely monitor for any residual impact. Service was fully restored by 15:34 ET.

This update is related to

Storefront is Degraded

Updated Tuesday, October 13th, 2009 at 2:54 PM ET
2009-10-13 at 18:54 UTC - Other time zones

Storefront customers may experience intermittent delays or timeouts when accessing services. Our teams are investigating.

As a temporary work-around, please reissue your requests.

Domains continue to resolve.

This update is related to

Domain Service is Degraded

Updated Tuesday, October 13th, 2009 at 2:52 PM ET
2009-10-13 at 18:52 UTC - Other time zones

Resellers may experience intermittent delays or timeouts when attempting to access OpenSRS Domains services via the API. Our Ops and Development teams are investigating. As a temporary work-around please reissue your requests.

(There was an similar issue with the Reseller Web Interface (RWI). This has since been resolved.)

All other OpenSRS services are fully available. Domains continue to resolve.

This update is related to