Managed Databases Control Plane and Connectivity
Incident
Report for DigitalOcean
Resolved
Our Engineering team has confirmed that this incident has been fully resolved.
If you continue to experience any issues with Managed Database Clusters please open a ticket with our support team. Thank you for your patience.
If you continue to experience any issues with Managed Database Clusters please open a ticket with our support team. Thank you for your patience.
Monitoring
Our Engineering team has implemented a fix to resolve the issue with our Managed Databases services. At this time, we're observing error rates returning to pre-incident levels and seeing operations such as create/fork/restore succeed. Trusted sources updates are also functioning normally, so connectivity to Database clusters from newly added resources to trusted sources is restored.
We are monitoring the situation closely and will post an update as soon as we confirm the issue is fully resolved.
We are monitoring the situation closely and will post an update as soon as we confirm the issue is fully resolved.
Update
Our Engineering team is still working on a fix for the issue with our Managed Databases service (excluding Mongo clusters). At this time, users may be impacted by:
- Errors/latency for creation, forking, and restoration of clusters
- 5xx errors on Managed Databases API endpoints
- Due to updates to trusted sources not taking effect, connections to Database clusters from newly added trusted sources will fail. This includes new Managed Kubernetes nodes, Droplets, and Apps using Databases.
As soon as we have further information, we'll provide another update.
- Errors/latency for creation, forking, and restoration of clusters
- 5xx errors on Managed Databases API endpoints
- Due to updates to trusted sources not taking effect, connections to Database clusters from newly added trusted sources will fail. This includes new Managed Kubernetes nodes, Droplets, and Apps using Databases.
As soon as we have further information, we'll provide another update.
Update
Our Engineering team continues to work towards resolving the issue impacting our Managed Databases Control Plane. At this time, they have also confirmed that new Managed Kubernetes nodes are unable to connect successfully to Managed Database clusters. Users may see connection time outs or failures from Kubernetes nodes to their Databases, excluding Mongo Databases.
Additionally, updates to trusted sources on Managed Database clusters are not being applied successfully, so any updates made will not take effect.
We'll provide another update as soon as possible.
Additionally, updates to trusted sources on Managed Database clusters are not being applied successfully, so any updates made will not take effect.
We'll provide another update as soon as possible.
Update
Our Engineering team is continuing to work on resolving the issue impacting Managed Database Control Plane. We can confirm that the Get, List, and Update operations are now functioning properly, while the creation function remains blocked. Additionally, we'd like to specify that MongoDB remains unaffected by this incident.
We apologize for the inconvenience and will share an update once we have more information.
We apologize for the inconvenience and will share an update once we have more information.
Identified
Our Engineering team has identified the cause of the issue impacting the Managed Database Cluster. During this time, users will continue to experience errors when working with Managed Databases such as creating, viewing, or updating Clusters.
We are actively working on a fix and will post an update as soon as additional information is available.
We are actively working on a fix and will post an update as soon as additional information is available.
Investigating
As of 11:22 UTC, our Engineering team is investigating an issue with the Managed Database Cluster. During this time, users may experience errors while creating Database clusters via the Cloud Control Panel and API requests.
We apologize for the inconvenience and will share an update once we have more information.
We apologize for the inconvenience and will share an update once we have more information.
This incident affected: Managed Databases (AMS3, BLR1, FRA1, LON1, NYC1, NYC2, NYC3, SFO2, SFO3, SGP1, SYD1, TOR1).