# Evidence and question insights

> Citations on every answer, the evidence panel with its passages, signed receipts that any party can verify, and question insights for each Octy profile.

Every Octy™ answer cites the passages it was written from and is checked against them before it is sent. The organisation can review what was answered, from which sources, and what each profile is asked. The evidence panel and signed receipts allow any party to check an answer afterwards.

## Citations and the grounding check

Every answer carries its **citations**: the documents and passages it was written from, with links where the source has one. The answer is checked against those passages before it is sent. A sentence the passages do not support is flagged or removed, as the profile specifies. Each answer reports whether it was grounded, and the count reaches LightHouse with the turn’s other figures. See [Evidence](/concepts/evidence/).

## The evidence panel

An **evidence panel** beside every answer, on every channel that can show one. For each source it shows the title and section, the passage exactly as indexed, the version or publication date of the source, and whether the answer cited it. The panel shows the passages Octy read as well as the answer.

## Signed receipts

Every answer carries a **signed receipt**: a small signed record of the answer, issued for the Octy profile that gave it. A receipt holds a digest of the question, a digest of the answer, the time, the Octy profile that answered, and a digest of each passage it was written from. It contains no question or answer text, no model name and no score. It is signed with Ed25519 under the issuer’s published key: Atlantic’s Octy key for an Atlantic-hosted profile, LightHouse’s Octy key for Octy in the portal, and the deployment’s own key for a profile that runs in a client deployment. Any party with the receipt and the published key can verify it.

A receipt establishes that a specific Octy profile gave a specific answer to a specific question, at a specific time, from specific passages. A changed word in the answer, or a substituted passage, fails verification. A receipt does not establish that the answer is correct. That judgement rests on the evidence panel and the cited sources.

## Verifying a receipt

Verification in the browser, from the evidence panel, and from the command line with `lighthouse-ai verify-answer`. Both check the signature against the issuer’s published key and compare the digests with the answer and passages shown.

## Audit

Each turn is recorded in the deployment and reported to LightHouse as operational telemetry: the time, the route taken, the cost, whether the answer was grounded, and the outcome. Content remains in the organisation’s account. A person is identified to LightHouse only by a keyed hash. See [The telemetry contract](/concepts/telemetry-contract/).

Changes to Octy’s configuration in a deployment are released through the release gate, like any agent change, and recorded there. Each change to an Octy profile is recorded in LightHouse™ with its revision and the changed fields. See [The audit](/octy/manage/#the-audit).

## Question insights

**Question insights** for each Octy profile, on its **Costs and questions** tab in LightHouse, for the last 7, 30 or 90 days.

Question insights show what a profile is asked: the questions by outcome (answered from retrieval, answered with a model, not covered, refused, or handed on), the top questions, the questions the knowledge did not cover, the sources cited most, and the volume per day. Where a GitLab or Jira connector is set up, a gap can be opened as an issue for the team that owns the knowledge. Test answers are excluded.

Question text is redacted before storage, carries no identifier of the person who asked, and is retained for twelve months. The storage location depends on where the profile runs:

| Profile runs on               | Question text storage                                | LightHouse view                                                                                                                                                                                                           |
| ----------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Atlantic-hosted               | In LightHouse, under the tenant’s row-level security | The redacted questions                                                                                                                                                                                                    |
| Client deployment, connected  | In the deployment’s own store                        | The redacted questions, read in place for this page only and not stored in LightHouse. Visible to `client_admin` and `content_viewer` only                                                                                |
| Client deployment, air-gapped | On site                                              | Counts, outcomes and clusters of equal questions, identified by a keyed hash. The text is on the site’s own Octy insights page, behind the site’s sign-in, where an administrator can match each cluster to its questions |

The keyed hash groups equal questions for counting. A guessed question cannot be tested against the hash without the deployment’s key.

Source: https://docs.atlanticagenticlabs.com/octy/evidence/