Skip to content

Datadog

Built. The engine reads Datadog monitors, incidents and postmortems, and Datadog connects from LightHouse.

You need the client_admin role. Go to Setup and admin → Connectors → Datadog and choose your site: us1, us3, us5, eu1, ap1 or us1-fed.

  1. Choose Approve in Datadog. Datadog shows its OAuth approval for LightHouse on your site (authorisation code with PKCE).
  2. Approve. Datadog returns you to LightHouse, which checks the grant.

If Atlantic’s Datadog application is not yet registered for your LightHouse, the card says so and offers the key route.

  1. Create an application key with only the scopes below at Organization settings → Application keys (app.datadoghq.com/organization-settings/application-keys on US1). You also need an API key.
  2. Paste both into the card and choose Connect.

LightHouse checks the pair with GET /api/v1/validate. A key with write scopes is refused.

lighthouse connect datadog will reuse DD_API_KEY and DD_APP_KEY from your environment, with your explicit yes. See The CLI.

Give the engine the two keys from your own secret store as DD_API_KEY and DD_APP_KEY, and add a source card:

- id: datadog-ops
purpose: Read monitors, incidents and postmortems for the operations agents.
executor: python
target: aal_engine.ingest.connector:run_card
overseer: sre-lead@example.com
budget_usd: 0
effect: internal
source:
kind: datadog
site: datadoghq.eu # the API host suffix: datadoghq.com, us3.datadoghq.com, datadoghq.eu, ...
kinds: [monitors, incidents, postmortems]
label: standard
requires:
reads: [monitors, incidents and postmortem notebooks]
reason: the operations agents answer from what was seen and learned

site here is the API host suffix (for example datadoghq.com or datadoghq.eu), not LightHouse’s site code. api_key_env and app_key_env change the variable names.

Scope Why
monitors_read Monitors
incident_read Incidents
notebooks_read Postmortem notebooks. Needed by the engine, but not in LightHouse’s approval list today: add it to the application key the engine uses
dashboards_read, logs_read_data, apm_read, metrics_read Asked for by LightHouse’s approval; the engine does not read dashboards, logs, APM or metrics today

With outbound telemetry chosen (below), LightHouse also asks for metrics_write and events_write, and accepts no other write scope.

The engine calls, all GET: /api/v1/monitor, /api/v2/incidents, and /api/v1/notebooks?type=postmortem.

  • For your deployment: in your own secret store, passed to the engine as DD_API_KEY and DD_APP_KEY. Rotate them there.
  • For the LightHouse connection: sealed under your tenant’s own key and never shown again.
Item What is kept
Monitors Message, query, type, state, tags and priority
Incidents State, severity, customer impact and the incident’s text fields
Postmortems The Markdown cells of postmortem notebooks; graphs are left out

Never read: dashboards, logs, traces and metrics.

How often: on the card’s schedule, reading what was modified since the last pull. A deleted monitor is removed from the corpus; incidents and postmortems are kept.

Outbound: LightHouse’s telemetry in your Datadog

Section titled “Outbound: LightHouse’s telemetry in your Datadog”

Optionally, LightHouse can send its own operational telemetry about your agents to your Datadog: metrics and events, nothing from your content. You choose it when connecting, or later with PUT /api/v1/connections/{cid}/outbound. LightHouse records the choice and asks for the two write scopes; the sending itself is Coming.

  • The source’s run note: written N, unchanged N, quarantined N, removed N.
  • The Datadog card in LightHouse: reads, items written and failures over the last 7 days.
  • Agents → Knowledge: documents from datadog.
  • In LightHouse: Disconnect clears the sealed credential and keeps the record. Datadog has no revocation LightHouse can call: revoke the application key, or remove the integration, in Datadog.
  • In your deployment: remove the card and the keys. Documents already written stay in the corpus until you delete them.