The Incident
On Monday July 31st 2017, we discovered a security bug that allowed one of our users to view other customer data in the newly deployed support portal. After validating and inspecting our logs, we have confirmed that only two records were exposed to one user, and the user who found this issue reported it to us immediately. It’s important to note that no customer billing data was compromised. We have personally notified the two customers whose accounts were impacted. We take your data privacy very seriously, and have mitigated the issue with the bugfix, and updated our internal change management policies to ensure this issue will not reoccur.
Timeline of Events
10:29 UTC: A user reports an information disclosure in the new support portal
12:24 UTC: DigitalOcean support discovers the report and escalates internally
12:50 UTC: Incident response starts, our security team disables the endpoint that contains the primary information disclosure while further investigation is performed
13:35 UTC: Full scope of information disclosure is now understood, support portal is taken offline
14:12 UTC: Investigation of access logs shows scope of disclosure
15:53 UTC: Final mitigations are in place for the specific incident
16:28 UTC: Support portal brought back online and incident resolved
Future Measures
Our new support portal is built on top of a well-known third party SaaS platform. This sits outside the traditional operational model for DigitalOcean engineers. Our post-mortem investigation led to the discovery of a blind spot in our operational practices and playbooks for these kinds of systems. Over the coming months, we will review and write new standard organizational procedures for the development, operations, and security of this class of system.
Our standard procedures made it unclear as to what the scope of any necessary security assessment for this system was. After conducting our internal analysis we are reviewing our internal procedures and making a determination as to what further security assessments are necessary to prevent similar situations from occurring in the future.
In Conclusion
We take any form of negative impact on your service seriously. Our goal is to constantly improve and we hope that our transparency and the details that we have shared help show you that.