LLM Audit Logging: What to Log, Redact, and Retain

Most teams shipping an LLM feature have monitoring before they have logging. Dashboards show request rate, p95 latency, and guardrail trigger counts, and everyone agrees the system is observable. Then a customer escalation arrives — your assistant told my user to do something it should never have said, on Tuesday — and the only honest answer available is a count of how many times a filter fired that day.

Monitoring answers is something wrong right now. Audit logging answers what exactly happened in this specific interaction, and can we prove it later. Those are different systems with different retention, different storage, and very different privacy exposure. This is a field-level guide to the second one: what to persist, what to strip before it lands, how long to hold it, and how to keep the log itself from becoming the breach.

Why LLM Logs Are a Harder Problem Than Application Logs

A conventional application log records a decision over structured inputs. You can log the whole request body and sleep fine, because the body is a form submission. An LLM log records free-form natural language on both sides of the boundary, and that changes three things at once.

The payload is the sensitive object. Users paste medical histories, contract clauses, source code, and credentials into chat boxes. A full-fidelity prompt log is, in practice, an unstructured PII store that nobody classified as one. It inherits every obligation your customer data store has, usually without inheriting any of its controls.

You cannot reproduce the event from the inputs. Sampling is stochastic, providers roll model versions, and system prompts change between deployments. Replaying the same prompt next week does not reconstruct what the model actually said. If the output was not captured at the time, the evidence is gone permanently — unlike a deterministic service, where the code plus the input is the record.

The log is also an attack corpus. Logging blocked prompts in full means building a searchable library of every working injection payload aimed at your system, sitting in a store that far more people can read than can reach production. That is a real consideration in the prompt injection detection workflow, where analysts genuinely need attack samples — but it argues for a separate, tightly scoped quarantine store rather than for dumping payloads into the general application log.

The Compliance Floor

If the application falls inside the EU AI Act’s high-risk scope, logging stops being an engineering preference. Article 12 requires that high-risk AI systems “technically allow for the automatic recording of events (logs) over the lifetime of the system,” with the recorded events sufficient to identify situations that may create risk, support post-market monitoring, and track system operation. Article 19 sets the retention floor: providers keep those automatically generated logs for a period appropriate to the intended purpose, and at least six months, unless other EU or national law — data protection law in particular — requires longer.

Two design consequences fall out of that pairing. First, the logging capability has to be built into the system rather than bolted on by an SRE afterwards, because Article 12 is a property of the system, not of the deployment. Second, six months of retention on records that contain user text is precisely the collision the redaction design has to resolve: you are required to keep the record and required to minimise the personal data in it. The resolution is almost always to keep the event for six months or more and the content for days.

The OWASP Top 10 for LLM Applications points at the same seam from the security side, though not under a logging heading. Sensitive information disclosure (LLM02) is what an over-broad prompt log becomes, and system prompt leakage (LLM07) is what a log that captures the system prompt verbatim becomes. Neither appears on that list as a logging problem, which is precisely why the logging design is where teams walk into both without noticing.

A Three-Tier Field Model

The workable pattern is to split every interaction into three tiers with different lifetimes and different access controls. Tier 1 is always written and kept long. Tier 2 is written conditionally and expires fast. Tier 3 is never written at all.

Tier 1 — always logged, long retention, low sensitivity

These are the fields that let you answer what happened without storing what was said. All of them are numeric, categorical, or opaque identifiers.

Field	Purpose
request_id, session_id, tenant_id	Correlation across services and per-customer scoping
timestamp_start, timestamp_end	Ordering, latency, and Article 12 period-of-use records
principal_id (pseudonymous)	Who invoked it, without carrying the identity itself
model_id, model_version, provider	Which artifact produced the output
system_prompt_hash, policy_version	Which configuration was in force
input_tokens, output_tokens, truncated	Cost, and detection of context-window pressure
guardrail_decisions[]	Per-layer verdict, reason code, and numeric score
retrieval_doc_ids[], retrieval_scores[]	Which corpus documents entered the prompt
tool_calls[]	Tool name, argument hash, and outcome
input_hash, output_hash	Correlation and duplicate detection without the text
finish_reason, error_code	Whether the model stopped normally

The hashes are what make this tier work. A salted hash of the prompt lets you group a coordinated campaign, match a support ticket to a stored interaction, and prove that a given text was or was not the input — all without holding the text. Use a keyed hash with a rotating salt, not a bare digest, or short inputs are trivially reversible by brute force.

The retrieval and tool-call fields are the ones teams most often skip and most often need. When a RAG answer goes wrong, the useful question is which documents were retrieved and at what score, not what the model said about them. Capturing document IDs turns a vague complaint into a corpus bug you can reproduce, which is why per-query retrieval logging belongs in any secure RAG architecture.

Tier 2 — conditionally logged, short retention, high sensitivity

Raw prompt and completion text goes here, and only under a sampling or triggering rule: a guardrail block, an explicit user report, an error, or a low-rate random sample for quality review. Store it in a separate system with its own access control and its own — much shorter — TTL, referenced from the Tier 1 record by request_id.

The OpenTelemetry GenAI semantic conventions encode exactly this distinction. Content attributes such as gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, and gen_ai.prompt.variable are all marked opt-in and flagged as carrying sensitive data, and the specification’s guidance is that instrumentation should not capture them by default — the operator has to explicitly turn them on, and instrumentations are expected to offer filtering and truncation. Those attributes currently sit at Development stability, so pin your instrumentation version and expect the shapes to move.

Adopting the convention names even for a homegrown pipeline is worth doing anyway: it makes the “does this field contain user content” question answerable by prefix rather than by tribal knowledge.

Tier 3 — never logged

• Full secrets or credentials, including ones the model reproduced from its context
• Raw regulated data (health, financial, biometric) unless a specific legal basis and a specific control set exist for it
• Blocked injection payloads in the general log — route those to a quarantine store instead
• Full retrieval document bodies, when the document IDs already point at the source of record

Redact on the Write Path, Not the Read Path

The single most common design error is treating redaction as a query-time concern: log everything, mask it in the viewer. That fails on every axis. The unredacted text is still on disk, still in backups, still in whatever index the log platform built, and still exported by anyone with raw API access to the logging vendor.

Redaction belongs inline, before persistence, in the same pipeline stage that already scans the output. The detectors you run for output filtering — entity recognition, secret patterns, structured identifiers — are the same detectors the redactor needs, so the marginal cost is a second consumer of an existing result rather than a new model call. Replace each match with a stable typed placeholder ([EMAIL:a91f], [CARD:3d02]) where the suffix is a keyed hash of the original. That preserves the two properties investigators actually use: you can still see the shape of the conversation, and you can still tell whether the same entity appeared twice.

Assume the redactor misses things. Recall on free-form text is never one. The short TTL on Tier 2 is what bounds the damage from the misses, which is why the tier split matters more than the redactor’s accuracy.

Retention Tiers That Satisfy Both Obligations

A workable default for a system in scope of Article 19:

• Tier 1 events: 13 months, in the metrics and events store. Comfortably above the six-month floor, and long enough to compare a quarter against the same quarter last year.
• Tier 2 content: 7 to 30 days, in a separate store, encrypted with a distinct key, access-logged, and deleted on schedule rather than on request.
• Quarantined attack payloads: 90 days, in a security-team-only store, with the associated user identifiers stripped at write time.
• Aggregated counters: indefinite. Daily rollups of trigger rates and score distributions carry no personal data and are what you need to detect the slow drift covered in monitoring LLM outputs in production.

Deletion has to be enforced by the storage layer’s lifecycle policy, not by a cron job someone wrote. A cron job that silently stops is indistinguishable from one that works, right up to the audit.

The Log Is Part of the Attack Surface

Two properties keep audit logs credible under adversarial conditions.

Append-only with integrity. Write-once storage with object-lock semantics, or a periodic hash chain where each batch commits the digest of the previous batch. Without it, “the logs show no such request” is an assertion rather than evidence, and an attacker who reaches the log store can rewrite the account of their own activity.

Independent access control. The service account that writes logs should not be able to read or delete them, and the humans who investigate incidents should hold a different grant from the humans who operate the model. A single role that can both produce and edit the record defeats the point of keeping one.

There is also a supply-side integrity question: the log tells you which model artifact produced an output, which is only meaningful if the artifact identity is trustworthy in the first place. That chain is the subject of securing the ML model supply chain.

Failure Modes Worth Checking For

• Logging the system prompt in full, on every request. It is usually the most competitively sensitive string in the application, and it is constant — log a version hash and store the prompt itself in configuration management.
• Unbounded fields. A filled 200k-token context window is roughly 800 KB of text before any JSON escaping or message structure, so a single record can approach a megabyte. Truncate at write time with an explicit truncated: true flag rather than discovering the cap in your log vendor’s billing.
• No abuse-relevant identifiers. Rate limiting and behavioural abuse detection depend on stable per-caller keys being present in the same records; without them, the analysis in rate limiting and abuse detection for AI APIs has nothing to join on.
• Streaming responses logged only on completion. A response abandoned mid-stream still reached the user.
• No log for the requests that never reached the model. An input rejected by a guardrail is the most security-relevant event in the system, and many pipelines return the refusal without recording anything.

If you are working out which of these controls your own deployment is actually missing, the Guardrail Gap Analyzer maps an application’s shape, trust boundary, and data sensitivity to the controls that apply and flags the ones you have not placed yet.

See also

• AI security benchmarks
• ML monitoring and drift
• LLM operations in production