rag-context-worker — business features

1. What it is

When an admin uploads a manual, this worker is what turns those bytes into something the rest of the platform can actually use: it sends the file to RAGFlow to be parsed and chunked, tracks how far that got so the admin sees a real progress bar, and afterwards answers questions of the form "give me the passages in this tenant's documents that are about forklift battery changes, within this token budget". Three services consume those passages over HTTP: authoring-service-v2, content-worker and suggest-pipeline-worker.

2. Feature map

flowchart LR
  ADMIN["Admin uploads a document"] --> AM[asset-manager]
  AM -. "upload.completed" .-> AV2[authoring-service-v2]
  AV2 -. "rag.indexing.requested" .-> F1["Index a document"]
  F1 --> RF["RAGFlow parse + chunk"]
  F1 -. "indexing progress / ready / failed" .-> AV2
  AV2 -. "rag.context.requested" .-> F2["Retrieve context for a topic"]
  F2 -. "context.ready / failed" .-> AV2
  F2 --> BLOB["context blob in S3"]
  BLOB --> HTTP["GET /contexts/{id}/stream"]
  HTTP --> CW["content-worker + suggest-pipeline"]

3. Features

Admin

Employee (mobile)

None.

Internal (other services)

Background

4. API reference

5. Async contracts

All four durables are single-binding by design — no queue group. The helper's long comment explains why: nats-py rejects a queue name that differs from the durable before it even reaches the server, so the four consumers were always created without a deliver group, and matching that reality is the honest fix. It is also why the Deployment is replicas: 1 with strategy: Recreate: a second pod binding the same durable raises on purpose (app/consumers/subscribe_helper.py:66-106).

Consumes

Publishes

Earlier scans reported this service as publish-free; it is not. It reports every indexing and retrieval outcome back to authoring-service-v2 — those six subjects are how an admin's upload progress and a draft's context ever become visible.

Background jobs

6. Data it owns

Sole writer of rag_context_db (ARCHITECTURE.md §3.4). No other service reads or writes it, and this worker writes no other service's tables — everything it shares travels as an event, an HTTP response or an S3 object.

Context text is not in Postgres: the blob lives in S3 and the row points at it, which is why the HTTP blob route can hand out a presigned redirect instead of proxying bytes.

7. Dependencies

flowchart LR
  AV2[authoring-service-v2] -. "indexing · context requested" .-> RAG[rag-context-worker]
  AV2 -- "HTTP H5: document meta · blob" --> RAG
  CW[content-worker] -- "HTTP H6: context stream" --> RAG
  SPW[suggest-pipeline-worker] -- "HTTP H10: context stream" --> RAG
  RAG -. "indexing + context progress/ready/failed · usage" .-> AV2
  RAG -- parse + query --> RF["RAGFlow (ragflow ns)"]
  RAG -- layout parse --> MU["Modal MinerU"]
  RAG -- "blobs · downloads" --> S3["S3 + Cloudflare CDN"]
  RAG -- "documents · contexts · chunks" --> DB[("rag_context_db")]

8. Dead-code verdicts

Every entry point with no in-repo caller. Deleting is a separate decision — see the hub roll-up.

All six published subjects have a named authoring-v2 consumer, and the indexing/retrieval consumers have named authoring-v2 publishers.

9. Sources

• rag-context-worker/README.md — the indexing/retrieval split (its "write to the draft record" line is stale; the blob is served over HTTP instead)
• spec/sdd-rag-context-worker.md — the design of record
• contracts/README.md, authoring-service.md, content-and-suggest-workers.md, asset-manager-service.md, chat-worker.md — the per-caller contracts, including the two events nobody publishes
• docs/docling-parser-rollout.md, docs/mineru-parser-rollout.md — parser rollout history (which parser backs indexing, and why)
• bug-hunt-reports/rag-context-worker.md — 12 findings; #1, #7, #8, #9 fixed; #2 (dataset-scoped retrieval), #3 (cancel acks early), #5 (spoofable tenant when the key is unset), #6 (SSRF on document download) open
• ARCHITECTURE.md §3.1 (spoke topology), §3.2 (H5, H6, H10 and the prefix conventions), §3.3 (AUTHORING_RAG and AUTHORING_USAGE ownership), §3.4 (rag_context_db), §3.6 (RAGFlow, MinerU, S3), §4.4 (Sentry DSN in a plaintext ConfigMap)
• app/api.py, app/consumers/, app/publishers/events_publisher.py, app/main.py, alembic/versions/, nats/streams/, k8s/
• Scan output tools/feature-docs/out/rag-context-worker.json