Subsea Fiber Faults Between Europe and Asia
Incident
Report for DigitalOcean
Resolved
Our Engineering team has continued to actively monitor the situation resulting from multiple subsea fiber faults in our Singapore (SGP1) region, and customers in Asia attempting to reach DigitalOcean resources hosted in our Amsterdam (AMS2, AMS3), London (LON1), and Frankfurt (FRA1) regions.
Our Support teams continue to surface customer escalations to our Networking team and work to shift to better/less latent routes will continue as well.
Repair of all cables from crews is still pending, and periodic network instability may continue to present itself during hours of high traffic, for multiple weeks until repairs are fully completed.
We will now close out this incident and provide any needed updates separately.
If users experience a disruption in service or have any questions, we invite them to submit a support ticket from within their account.
Thank you for your patience and understanding throughout this incident.
Our Support teams continue to surface customer escalations to our Networking team and work to shift to better/less latent routes will continue as well.
Repair of all cables from crews is still pending, and periodic network instability may continue to present itself during hours of high traffic, for multiple weeks until repairs are fully completed.
We will now close out this incident and provide any needed updates separately.
If users experience a disruption in service or have any questions, we invite them to submit a support ticket from within their account.
Thank you for your patience and understanding throughout this incident.
Update
Our Engineering team continues to work with upstream providers to resolve network connectivity issues that are occurring in multiple regions. As a result of multiple subsea fiber faults between Europe and Asia regions, the issues appear to be a direct consequence of congestion resulting in packet loss and increased latency connecting to the SGP1 region. Customers in the Asian region are having difficulty reaching DigitalOcean resources located in Amsterdam (AMS2, AMS3), London (LON1), and Frankfurt (FRA1).
We have no ETA for the upstream issues being restored but we will communicate any relevant information as we have it. We apologize for the inconvenience.
We have no ETA for the upstream issues being restored but we will communicate any relevant information as we have it. We apologize for the inconvenience.
Identified
We've confirmed the degraded network performance in our first update is due to a cable cut, which occurred at approximately 18:00 UTC, April 19. This cable is located just west of Singapore and impacts the cabling system connecting 17 countries from Singapore to Europe. Users can read more about the cable here: https://www.submarinenetworks.com/en/systems/asia-europe-africa/smw5
We continue to monitor this situation and progress of the repair. Our Engineering team will also continue to tune for performance and alternate routes where possible, until the issue is resolved.
If you have any questions, please open a Support ticket from within your account.
We continue to monitor this situation and progress of the repair. Our Engineering team will also continue to tune for performance and alternate routes where possible, until the issue is resolved.
If you have any questions, please open a Support ticket from within your account.
Investigating
Due to multiple simultaneous subsea fibre cuts between Europe and Asia, we are observing degraded performance for European customers attempting to reach DigitalOcean resources hosted in our Singapore (SGP1) region, and Asian customers attempting to reach DigitalOcean resources hosted in our Amsterdam (AMS2, AMS3), London (LON1), and Frankfurt (FRA1) regions.
Engineers are attempting to improve performance where possible, but as this issue is largely out of our control we do not expect a substantial improvement in performance until one of the fibre cuts is repaired.
We apologize for any inconvenience.
Engineers are attempting to improve performance where possible, but as this issue is largely out of our control we do not expect a substantial improvement in performance until one of the fibre cuts is repaired.
We apologize for any inconvenience.
This incident affected: Networking (AMS2, AMS3, FRA1, LON1, SGP1).