Managed Databases Control Plane
Incident
Report for DigitalOcean
Resolved
Our Engineering team has confirmed that this incident has been fully resolved.
We appreciate your patience throughout this process and if you continue to experience problems, please open a ticket with our support team for further review.
We appreciate your patience throughout this process and if you continue to experience problems, please open a ticket with our support team for further review.
Monitoring
Our Engineering team has implemented a fix to fully resolve the issue with our Managed Databases services. At this time, users should no longer face issues creating, resizing, forking, or updating trusted sources for PostgreSQL, MySQL, Redis, and Kafka clusters, in any region, as well as Mongo clusters in NYC regions.
We are monitoring the situation closely and will post an update as soon as we confirm the issue is fully resolved.
We are monitoring the situation closely and will post an update as soon as we confirm the issue is fully resolved.
Update
Our Engineering team has been able to partially mitigate impact from this incident. Creating, resizing, forking, and updates to trusted sources of PostgreSQL, MySQL, Redis, and Kafka clusters, in all regions, is now working as expected. Operations previously attempted on clusters, such as firewall rule attempts, will need to be resubmitted or have a relevant event (i.e. k8s node changes, tag events, etc.) occur from now on. Users should start to see recovery and be able to initiate new operations normally.
Operations for Mongo clusters remain affected at this time.
Operations for Mongo clusters remain affected at this time.
Update
Our Engineering team continues work to address the root cause of this issue and remediate customer impact.
We apologize for the continued interruption to the Managed Databases Control Plane and will provide another update as soon as we have more information.
We apologize for the continued interruption to the Managed Databases Control Plane and will provide another update as soon as we have more information.
Identified
During the course of investigation, our Engineering team has identified additional operations are impacted by this incident.
Users may experience errors with creating, resizing, or updating trusted sources for PostgreSQL, MySQL, Redis, and Kafka clusters, in all regions, as well as Mongo clusters in NYC regions.
Due to updates to trusted sources being impacted, connections to Database clusters from newly added trusted sources will fail. This includes new Managed Kubernetes nodes, Droplets, and Apps using Databases.
Our team is working to deploy a fix and we will provide another update soon.
Users may experience errors with creating, resizing, or updating trusted sources for PostgreSQL, MySQL, Redis, and Kafka clusters, in all regions, as well as Mongo clusters in NYC regions.
Due to updates to trusted sources being impacted, connections to Database clusters from newly added trusted sources will fail. This includes new Managed Kubernetes nodes, Droplets, and Apps using Databases.
Our team is working to deploy a fix and we will provide another update soon.
Investigating
Our Engineering team is investigating an issue impacting new Managed Database creations. During this time, users may receive failures while creating new Managed Database clusters across all regions. We apologize for the inconvenience and will share an update once we have more information.
This incident affected: Managed Databases (AMS3, BLR1, FRA1, LON1, NYC1, NYC2, NYC3, SFO2, SFO3, SGP1, SYD1, TOR1).