DigitalOcean Status

Continuation of Global Networking Maintenance

Update - The maintenance effort continues to progress. At this time, we have completed work in the following regions:
FRA1 and SFO3

We are approximately 70%-75% complete in these regions:
NYC1, TOR1, and BLR1

We are approximately 45% complete in all other regions across our fleet.

In total, we are approximately 65% complete in this maintenance effort.

We will continue to provide updates here as incrementally complete this effort.

As a reminder, if you have any questions or concerns about this maintenance, please open a Support ticket from within your account.
Oct 22, 15:23 UTC

In progress - The maintenance effort continues to progress. At this time, we are approximately 65% complete across all regions. Our Engineering team has seen significant unexpected issues with this maintenance rollout, and at this time we will be proceeding more slowly and with a targeted approach in our FRA1 region. This will allow our team to continue the rollout, but help ensure that Droplets are not losing networking capabilities as a result of the rollout.

We will continue to provide updates here as incrementally complete this effort.

As a reminder, if you have any questions or concerns about this maintenance, please open a Support ticket from within your account.
Oct 12, 17:45 UTC

Scheduled - Our Engineering team will be continuing the previous maintenance here: https://status.digitalocean.com/incidents/0gl7pplrsd96

This is in order to reduce bottlenecks that limit network throughput and stability under certain user workloads on DigitalOcean Hypervisors.

We expect this effort to be ongoing from today, October 5, 2020, to October 15, 2020, and will be rolling out incrementally across our fleet. As of today, we are about 25% complete across our fleet.

Expected Impact:

There should be minimal downtime for Droplets on each affected hypervisor during the rollout. The expectation is that most users will not observe any impact, though some users with a sustained high-packets-per-second or high-throughput workload will see a small number of dropped packets.

Should users have questions about specific Droplets and this maintenance, we ask that they open a Support ticket within their account for assistance. Thank you for your patience throughout this process.

Regions Operational

Global Operational

AMS2 Operational

AMS3 Operational

BLR1 Operational

FRA1 Operational

LON1 Operational

NYC1 Operational

NYC2 Operational

NYC3 Operational

SFO1 Operational

SFO2 Operational

SFO3 Operational

SGP1 Operational

TOR1 Operational

Services Operational

API Operational

App Platform Operational

Billing Operational

Block Storage Operational

Cloud Control Panel Operational

Cloud Firewall Operational

Community Operational

DNS Operational

Droplets Operational

Event Processing Operational

Kubernetes Operational

Load Balancers Operational

Managed Databases Operational

Monitoring Operational

Networking Operational

Spaces Operational

Support Center Operational

Spaces CDN Operational

Operational

Degraded Performance

Partial Outage

Major Outage

Maintenance

Past Incidents

Oct 24, 2020

No incidents reported today.

Oct 23, 2020

Issues With New App Platform Builds and Spaces Creates

Resolved - Our engineering team has resolved the issue impacting creation of new Spaces buckets and new App Platform static site builds, and all services should now be operating normally. We apologize for any inconvenience, and thank you for your patience throughout this process. If you should encounter any further problems, please reach out to our support team right away.
Oct 23, 20:37 UTC

Monitoring - Our engineering team is monitoring an issue which impacted creation of new Spaces buckets and new App Platform static site builds across all regions between 02:58 UTC and 15:58 UTC today, October 23. During this time, users may have experienced errors with new static site builds on App Platform, as well as when creating new Spaces buckets. Our engineering team has deployed a fix, and are actively monitoring the situation. Thank you for your patience, and will share more information as soon as it's available.
Oct 23, 16:15 UTC

Issues with Spaces service

Resolved - Our Engineering team has resolved the issue impacting Spaces availability in our AMS3 region, and all services should now be functioning normally. Thanks again for your patience throughout this process. If you should encounter any further problems, then please open a ticket with our support team right away.
Oct 23, 13:26 UTC

Monitoring - Our Engineering team has deployed a fix for issues impacting Spaces availability in our AMS3 region, and is monitoring the situation closely. Thanks again for your patience, and we will provide another update once the matter is fully resolved.
Oct 23, 12:45 UTC

Identified - Our Engineering team has identified the cause of issues impacting Spaces availability in our AMS3 region, and are working on deploying a fix. Thank you for your patience, and we will provide more information once it's available.
Oct 23, 12:31 UTC

Investigating - Our Engineering team is investigating an issue with Spaces in our AMS3 region. During this time you may experience issues while working with the Spaces service. We apologize for the inconvenience and will share an update once we have more information.
Oct 23, 11:36 UTC

Oct 22, 2020

Issues with provisioning Droplets through Marketplace

Resolved - Our engineering team has resolved the issue impacting creation of Droplets via the Marketplace, and all services should now be functioning normally. Thanks again for your patience throughout this process. If you should encounter any further problems, then please open a ticket with our support team right away.
Oct 22, 17:20 UTC

Monitoring - Our engineering team has deployed a fix for the issue impacting creation of Droplets via the Marketplace, and is actively monitoring the situation. Thanks again for your patience, and we will provide another update once the matter is fully resolved.
Oct 22, 16:43 UTC

Identified - Our engineering team has identified the cause of an issue impacting creation of Droplets via the Marketplace, and is working on deploying a fix. Thank you for your patience, and we'll share an update once more information is available.
Oct 22, 14:08 UTC

Investigating - Our engineering team is investigating an issue with creating Droplets via the Marketplace across all regions. During this time, you may experience errors or problems when creating Droplets through the Marketplace. We apologize for the inconvenience and will share an update once we have more information.
Oct 22, 09:32 UTC

Oct 21, 2020

Issues with a Subset of Hypervisors in AMS3

Resolved - Our Engineering team has resolved the issue with a small number of servers in our AMS3 region. Accessing Droplets, Kubernetes master nodes, load balancers, and their related services should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.
Oct 21, 04:40 UTC

Monitoring - Our engineers have applied new fixes, and have observed that all Droplets have remained stable, however they will continue to monitor the situation. We will post an update as soon as the issue is fully resolved.
Oct 21, 01:36 UTC

Identified - Our engineers have noticed that there are still ongoing issues with AMS3 servers. Our engineers will continue working towards a solution. We will post an update as soon as we have applied a new fix.
Oct 20, 23:23 UTC

Monitoring - Our engineers have implemented a fix to resolve the issue with the servers in our AMS3 region and are monitoring the situation. We will post an update as soon as the issue is fully resolved.
Oct 20, 22:41 UTC

Identified - Our Engineering team has identified the cause of the issue with the servers in our AMS3 region and are actively working on a fix. We will post an update as soon as additional information is available.
Oct 20, 21:35 UTC

Investigating - Our engineering team is investigating an issue with a small number of servers in our AMS3 region. During this time you may be unable to access some Droplets, Kubernetes master nodes, load balancers, and their related services. We apologize for the inconvenience and will share an update once we have more information.
Oct 20, 21:18 UTC

App Platform Clusters Using Custom Domains Experiencing Delays in Changes Deployed

Resolved - Our engineering team has resolved the issue with App Platform's delays to deploying changes . App Platform should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.
Oct 21, 04:36 UTC

Monitoring - Our engineers have implemented a fix to resolve the issue with App Platform and are monitoring the situation. We will post an update as soon as the issue is fully resolved.
Oct 20, 22:58 UTC

Identified - Our Engineering team has identified the cause of the issue with App Platform's delays to deploying changes and is actively working on a fix. We will post an update as soon as additional information is available.
Oct 20, 20:49 UTC

Oct 20, 2020

Oct 19, 2020

Unable to Create Team Accounts

Resolved - Our engineers have determined that from October 19th 2020 17:02 UTC until October 20th 2020 16:35 UTC, some users were unable to create Team accounts from within the Cloud panel. During this time, some users experienced issues proceeding beyond Step 2 (Billing Info) of creating a Team account. Our team was able to take action to resolve the issue, and all services are now functioning normally. If you are still experiencing issues or have additional questions then please open a Support ticket within your account
Oct 19, 17:00 UTC

Oct 18, 2020

Issues with Attachments in Support Tickets.

Resolved - The issue with ticket attachments in our support system has now been fully resolved. If you continue to see issues with your attachments please let our support team know and they will be happy to assist you.
Oct 18, 00:43 UTC

Monitoring - Our engineering team has implemented a fix for the issue with attachments in the support system and are monitoring the situation. You should now be able to add attachments to your tickets normally.
Oct 17, 23:50 UTC

Investigating - We are currently investigating an issue with our support ticket system causing attachments to not process correctly. We hope to have this issue resolved soon and apologize for the inconvenience.
Oct 17, 23:43 UTC

Oct 17, 2020

Support Ticket/Portal Issues

Resolved - Our Engineering team has resolved the issue with our Cloud Support Portal, and access to the portal should be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.
Oct 17, 12:04 UTC

Monitoring - Our Engineering team has implemented a fix to resolve the issue with our Cloud Support Portal and is monitoring the situation. We will post an update as soon as the issue is fully resolved.
Oct 17, 11:33 UTC

Identified - Our Engineering team has identified the cause of the issue with our Cloud Support Portal and is actively working on a fix. We will post an update as soon as additional information is available.
Oct 17, 11:22 UTC

Investigating - Our Engineering team is investigating an issue with our Cloud Support Portal. During this time, users may experience issues accessing the Cloud Support Portal We apologize for the inconvenience and will share an update once we have more information.
Oct 17, 10:15 UTC

Oct 16, 2020

Networking issues in LON1 region

Resolved - Our engineers have resolved the issue impacting network connectivity in our LON1 region, and all services should now be operating normally. We apologize for any inconvenience, and we appreciate your patience throughout this process. If you should notice any further problems, then please reach out to support immediately.
Oct 16, 20:03 UTC

Monitoring - Our engineers have implemented a fix for the issue impacting network connectivity in our LON1 region, and are monitoring the situation closely. We will post another update once we're confident that the issue is fully resolved.
Oct 16, 19:42 UTC

Identified - Our engineers have identified an issue impacting network connectivity for a subset of customers in our LON1 region, and are actively working on a fix. During this time, users may experience intermittent packet loss or timeout errors when accessing Droplets and other resources (Kubernetes, Managed Databases, Load Balancers) in the LON1 region. We apologize for any inconvenience, and we will share more information as soon as it's available.
Oct 16, 18:37 UTC

Oct 15, 2020

No incidents reported.

Oct 14, 2020

Spaces issue in SGP1

Resolved - Our Engineers have resolved the issue impacting Spaces in our SGP1 region, and all services should now be operating normally. We apologize for the inconvenience, and we appreciate your patience throughout this process. If you should notice any further problems, then please reach out to Support for assistance.
Oct 14, 11:09 UTC

Monitoring - Our Engineering team has implemented a fix to resolve the issue with Spaces in our SGP1 region and is monitoring the situation. We will post an update as soon as the issue is fully resolved.
Oct 14, 08:57 UTC

Identified - Our Engineering team has identified the cause of the issue with Spaces in our SGP1 region and is actively working on a fix. We will post an update as soon as additional information is available.
Oct 14, 08:07 UTC

Investigating - Our Engineering team is investigating an issue with Spaces in our SGP1 region. During this time you may experience timeouts and slow requests while accessing Spaces. We apologize for the inconvenience and will share an update once we have more information.
Oct 14, 07:35 UTC

Oct 13, 2020

No incidents reported.

Oct 12, 2020

Cloud Control Panel Connectivity

Resolved - Our Engineering team has resolved the issue with our Cloud Control Panel. All aspects of https://cloud.digitaloocean.com should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.
Oct 12, 14:17 UTC

Monitoring - Our Engineering team has identified the issue and implemented a fix to resolve the issue with our Cloud Control Panel. We are currently monitoring the situation and will post an update as soon as the issue is fully resolved.
Oct 12, 13:46 UTC

Investigating - Our Engineering team is investigating an issue with our Cloud Control Panel. During this time, users may experience errors reaching https://cloud.digitalocean.com, issues while signing in there, as well as issues navigating the Cloud Control Panel. We apologize for the inconvenience and will share an update once we have more information.
Oct 12, 13:25 UTC

Oct 11, 2020

No incidents reported.

Oct 10, 2020

No incidents reported.

Subscribe to Incident