Issues with data ingesting and alerting

Incident Report for Datadog US3

Resolved

This incident has been resolved.
Posted Jul 11, 2024 - 22:44 EDT

Update

Remediation efforts continue.

RUM and Application Vulnerability Management are operational again.

Cloud SIEM and Cloud Security Management, as well as alerting off these data types, continue to be impacted.

We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 22:11 EDT

Update

Remediation efforts continue.

Profiling is operational again.

RUM, Cloud SIEM, Cloud Security Management, and Application Vulnerability Management, as well as alerting off these data types, continue to be impacted.

We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 21:42 EDT

Update

We continue to deploy fixes and are monitoring the results. We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 21:09 EDT

Update

We continue to deploy fixes and are monitoring the results.

RUM, Profiling, Cloud SIEM, Cloud Security Management, and Application Vulnerability Management, as well as alerting off these data types, continue to be impacted.

We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 20:26 EDT

Update

We continue to deploy fixes and are monitoring the results. We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 19:45 EDT

Update

APM is operational at this time, and alerting based on APM data has also resumed.

RUM, Profiling, Cloud Security Management, and Application Vulnerability Management, as well as alerting off these data types, continue to be impacted.

We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 18:38 EDT

Monitoring

We have deployed a fix and are monitoring the results.

Certain data types (Logs, NPM, and Synthetics) are operational again, and alerting from those types has also resumed.

APM, RUM, Profiling, Cloud Security Management, and Application Vulnerability Management, as well as alerting off these data types, continue to be impacted.

We will provide another update once the issue is fully resolved.
Posted Jul 11, 2024 - 18:04 EDT

Investigating

We are investigating an issue with ingesting data which began around 20:40 UTC. As a result, data from Log Management, APM, Synthetics, Profiling, RUM, CSM, and NPM is delayed. Additionally, monitors derived from this data are delayed.
Posted Jul 11, 2024 - 17:18 EDT
This incident affected: APM, Application Vulnerability Management, Cloud Network Monitoring, Cloud SIEM, Log Management, Monitors, Profiling, RUM, Synthetics, and Workload Protection.