Skip to content

Answer Formatting — Implementation

Plan: docs/95_plans/answer-formatting.md.

What changed

File Change
app/api/routers/chat.py New _sse_event() helper: multi-line data emitted as one data: line per text line. Used by the token events (RAG and clarification paths) and error events.
app/services/llm.py _build_system_prompt: "Answer format (Markdown)" block added directly above the language/script line, which stays last.
tests/routers/test_chat_streaming.py _reassemble_sse_tokens spec-compliant client mirror; byte-identity tests for streamed markdown (RAG + clarification paths); wire-format invariant (no bare SSE lines); non-streaming parity pin.
tests/services/test_llm.py TestAnswerFormattingContract: formatting block present, RTL same-structure clause, block ordered above the final language line (with and without persona/exercise blocks).

Wire format

Before (newlines corrupted the frame):

event: token
data: ...fin de phrase [1].

Exemple suivant...

After (SSE spec multi-line data; consumers rejoin with \n):

event: token
data: ...fin de phrase [1].
data:
data: Exemple suivant...

Verified consumers (no changes needed, both already join data: lines with \n):

  • gateway src/chat/sse-accumulator.ts
  • gateway promptfoo/transforms/response.js

Decisions

Decision Reason
Fix framing in the emitter, not the consumers Consumers are already SSE-spec-compliant; the producer was the only spec violation.
Only token/error events use _sse_event JSON-serialized events cannot contain raw newlines (json.dumps escapes them); keeps the diff minimal.
Formatting contract as a separate prompt block Concurrent work owns the numbered grounding contract; the language line must keep final-position salience.

Caveats

  • The prompt pins prove the instruction is present, not that the model obeys it; structure quality is confirmed by live probes / promptfoo evals.
  • Clients must render Markdown (website PR); plain-text clients still see readable line-broken text.