notification-worker — business features

Turns platform events into phone push notifications and an in-app inbox · ← platform hub · entry points verified against tools/feature-docs/out/notification-worker.json

1. What it is

notification-worker is the platform's delivery layer: other services announce that something happened — a module was assigned, a reminder came due, a streak is about to break — and this service decides who should hear about it, writes the message into that person's in-app inbox, and pushes it to their phone through Firebase. It also serves the inbox itself (history, unread badge, read/opened tracking), the admin notification centre, and the device registry that records which phones a person can be reached on.

It is the only Python service in the repo that is not FastAPI: main.py starts a plain aiohttp application (main.py:22-28) and then hands the process to NotificationWorker.run(), so there is no OpenAPI schema, no dependency injection and no /health route. Its health endpoints are /healthz and /readyz (app/routes/api.py:743-744) — different paths from every other service on the platform, and the k8s probes are wired to those two paths specifically (k8s/deployment.yaml:77-88).

Who uses itEmployee mobile app (inbox + device registry) · admin web app (admin inbox, notification settings) · assignment-service, users-auth-service (internal HTTP) · assignment-service, micro-learning-service-v2, feed-service, authoring-service-v2, users-auth-service (NATS publishers)
Runtimeaiohttp 3.9 (not FastAPI) + 9 JetStream pull consumers on one event loop · Deployment notification-worker, 1 replica (k8s/deployment.yaml:9) · health /healthz, /readyz
DatabasePostgres notifications on users-auth-postgres (k8s/es-db-notification-worker.yaml:37); sole writer (ARCHITECTURE.md §3.4)
Redisusers-auth-redis db 1 (k8s/configmap.yaml:7-9) — daily-cap counters, preference/admin/active-assignment caches
NATS streamsConsumes MICRO_LEARNING (9 durables); publishes auth.admin.alert onto AUTH. It also ships its own MICRO_LEARNING and CHAT_MESSAGES stream definitions (k8s/nats-screams-cm.yaml) that no initContainer runs — see §8
External APIsFirebase Cloud Messaging (FCM), via firebase-admin with a 10s per-call timeout (app/services/notification_service.py:192-198)
Entry points15 HTTP routes · 11 NATS consumer registrations (10 distinct subjects) · 1 published subject · 2 background jobs

2. Feature map

flowchart LR
  MOB["Employee app (off-repo)"] --> INBOX["Devices + notification inbox"]
  ADM["Admin web (off-repo)"] --> ADMIN["Admin inbox + email settings"]
  UA["users-auth-service"] --> INT["Push reachability + GDPR export"]
  AS["assignment-service"] --> TRAIN["Assigned, reminder, overdue, escalation"]
  ML["micro-learning-service-v2"] --> GAME["Streak + leaderboard nudges"]
  FS["feed-service"] --> FEEDN["Feed announcements"]
  INBOX --> HTTP["15 aiohttp routes"]
  ADMIN --> HTTP
  INT --> HTTP
  TRAIN --> NATS["9 MICRO_LEARNING durables"]
  GAME --> NATS
  FEEDN --> NATS
  HTTP --> STORE["notifications + user_devices"]
  NATS --> STORE
  NATS --> FCM["Firebase FCM push"]

3. Features

Admin

Admin notification inbox live

admin

An admin opens the bell in the admin web app and sees what happened across their tenant: learners who joined, modules that finished generating, assignments that were rolled out, training that went critical. Items carry tags and a deep link so the admin can jump straight to the thing that needs attention, and the unread count drives the bell badge. Admins have no phones registered here, so an admin notification is inbox-only — the row itself is the delivery (app/services/admin_notification_service.py:1-8).

Entry points
GET /v1/admin/notifications · POST /v1/admin/notifications/read-all · POST /v1/admin/notifications/{notification_id}/read
Touches
notifications (rows with audience='admin'), admin_notification_preferences
Related
docs/fe-admin-notification-settings.md, docs/admin_notification_preferences.md, New-Design/SDD-admin-notification-centre.md
Evidence
app/routes/api.py:765-777 · scope gate app/routes/api.py:377-394 · frontend contract docs/fe-admin-notification-settings.md:175-176

Admin notification settings and email mirror live

admin

Each admin chooses which categories of team notification they want — new learners, module generation, critical training alerts, training completions, the overdue digest — and, for the two time-sensitive categories, whether they also want an email. Muting only affects new notifications; it never clears the existing inbox. Email is never allowed without the in-app notification, so a "email on, in-app off" request is rejected rather than silently accepted. When an opted-in admin gets an email-capable notification, this service publishes one auth.admin.alert event per recipient and otp-worker sends the mail through Mailgun.

Entry points
GET /v1/admin/notifications/preferences · PUT /v1/admin/notifications/preferences · publishes auth.admin.alert
Touches
admin_notification_preferences · NATS AUTH stream · otp-worker → Mailgun
Related
docs/fe-admin-notification-settings.md, kill switch ADMIN_EMAIL_ALERTS_ENABLED (app/core/config.py:153)
Evidence
app/routes/api.py:766-771 · categories app/services/admin_preferences_service.py:36-85 · publish app/services/admin_notification_service.py:261 · consumer users-auth-service/app/otp_worker/processor.py:161-162

Rollout finished / rollout failed alerts live

admin

An admin assigns a module to a group and walks away. When assignment-service has finished writing out every recipient, the admin gets an inbox item saying the rollout landed and how many learners it reached; if the expansion failed, they get an alert saying the learners were not assigned. These two are rendered here rather than by the producer, because the expansion events carry no admin-ready wording (app/services/admin_notification_service.py:15-18).

Entry points
NATS training.assignments.ten.*.assignment.expansion_completed · training.assignments.ten.*.assignment.expansion_failed
Touches
notifications (audience='admin'), users-auth tenant-admins lookup
Related
Rollout alerts are deliberately not mutable in the settings screen (docs/fe-admin-notification-settings.md:164-166)
Evidence
app/services/admin_notification_service.py:602-612 · publisher assignment-service/app/core/nats_client.py:228,248 called from assignment-service/app/workers/expansion_worker.py:391,495,525

Producer-rendered admin envelopes live

admin

Other services post ready-made admin messages onto one shared subject branch and this service just files them: a learner joined the tenant, a module finished (or failed) generating in authoring, an assignment was fully completed by everyone, the daily overdue-learners digest, and the GDPR export a back-office admin requested. Recipients who muted that category are dropped at the single choke-point all admin paths share.

Entry points
NATS training.notifications.ten.*.admin.>
Touches
notifications (audience='admin'), admin_notification_preferences
Related
New-Design/PLAN-admin-notif-centre-tier1.md, category map app/services/admin_preferences_service.py:75-81
Evidence
app/services/admin_notification_service.py:596-600 · publishers assignment-service/app/services/admin_notifications.py:59, authoring-service-v2/app/publishers/admin_notifications.py:221, users-auth-service/app/routes/mobile_profile.py:216, users-auth-service/app/services/gdpr_export.py:347

Employee (mobile)

Device registration live

employee

On sign-in the app hands over its Firebase token and platform so the person becomes reachable by push; on sign-out it hands it back and the device stops receiving anything. A token that Firebase later rejects as dead is deactivated automatically on the next send, so a reinstalled or wiped phone stops costing delivery attempts. Registering someone else's token is refused, and un-registering is scoped to the caller's own tenant and user (a cross-tenant device-disable hole was fixed — bug-hunt-reports/notification-worker.md #1).

Entry points
POST /v1/devices/register · POST /v1/devices/unregister
Touches
user_devices · Firebase token validity
Related
docs/ARCHITECTURE.md:583-650; dead-token pruning app/services/notification_service.py:116-132
Evidence
app/routes/api.py:745-746 · register caller global-configs/management_scripts/register_device_keep.py:132 · scoped unregister app/routes/api.py:320-326

Notification inbox: history, unread badge, read and opened live

employee

The app shows a bell with an unread count and a two-tab screen: New and All. Tapping a card marks it opened (which implies read) and navigates to the deep link; staying on the screen for five seconds marks everything read and clears the badge. Because every notification is written to the inbox before the push is attempted, a person who was offline, in quiet hours or over their daily cap still finds the message waiting for them.

Entry points
GET /v1/notifications/history · POST /v1/notifications/{notification_id}/open · POST /v1/notifications/read-all · POST /v1/notifications/{notification_id}/read
Touches
notifications (read_at, opened_at)
Related
docs/flutter_notifications_guide.md, spec/oper_notification_history_sdd.md, docs/notification_deeplinks.md
Evidence
app/routes/api.py:755-762 · frontend contract docs/flutter_notifications_guide.md:20-23 · tracking roll-up to-be-reviewed/README.md:60-66

Training notifications: assigned, cancelled, completed live

employee

An admin assigns a module and the learner gets a push within seconds — "you've been assigned X" — plus an inbox row that deep-links into the module. The same channel carries cancellation and completion messages. The learner's own notification preferences decide whether the push goes out; the inbox row is written either way, and a successful push stamps the row as delivered so the platform can tell "sent" from "merely created".

Entry points
NATS training.notifications.ten.*.assignment.> (durable notification_worker)
Touches
notifications, user_devices, FCM, users-auth preferences
Related
docs/assignment_notification_contract.md, docs/notification_deeplinks.md
Evidence
app/services/notification_service.py:1823-1846 · publisher assignment-service/app/core/nats_client.py:286 (types created/cancelled/completed at assignment-service/app/services/notification_service.py:159,251 and assignment-service/app/services/assignment_progress_service.py:1155)

Assignment reminders and overdue nudges live

employee

A learner who has not finished required training gets a daily nudge, and a harder one once the due date has passed. Reminders respect the learner's quiet hours (a suppressed push is queued and sent when the window ends) and a per-person daily ceiling of five pushes; anything over the ceiling still lands in the inbox, marked capped, rather than vanishing. This whole feature was silently dead until the consumer filter was corrected — the old filter required one token more than the publisher ever emits.

Entry points
NATS training.assignments.reminders.ten.* (durable notification_worker_reminders_v2)
Touches
notifications, deferred_notifications, Redis daily-cap counters, FCM
Related
filter fix _to_filter_subject (app/services/notification_service.py:99-113); fix log to-be-reviewed/README.md #1
Evidence
app/services/notification_service.py:1255-1291 · publishers assignment-service/app/services/reminder_scheduler.py:496 and assignment-service/app/services/manual_reminder_service.py:204

Escalation: the final nudge live

employee

When a critical assignment is overdue and the learner still has not started, assignment-service emits a one-shot escalation. By product decision this is a last nudge to the employee, not a manager escalation: it is persisted to the inbox, delivery-tracked, and deliberately bypasses quiet hours and the daily cap — but it still honours a global push opt-out. The same event also fans out an inbox item to the tenant's admins under the "critical training alerts" category, which is email-capable.

Entry points
NATS training.assignments.escalations.ten.* (durable notification_worker_escalations)
Touches
notifications (learner + admin rows), FCM, auth.admin.alert email mirror
Related
product decision recorded in app/core/config.py:72-80 and to-be-reviewed/notification-worker.md:26-30
Evidence
app/services/notification_service.py:1454-1477 · admin fan-out app/services/admin_notification_service.py:516-568 · publisher assignment-service/app/services/reminder_scheduler.py:529

Streak-at-risk reminder live

employee

A learner on a learning streak who has not trained today gets an evening reminder before the streak breaks. It is deliberately polite: the push only fires inside the learner's own local evening window (18:00–22:00 by their timezone), an event that arrives earlier is held until that window opens, and one that arrives later is dropped from push while still landing in the inbox for the next app open. Quiet hours always win for streaks — there is no admin override.

Entry points
NATS learning.streak.at_risk.ten.* (durable notification_worker_streak_v2)
Touches
notifications, deferred_notifications, FCM
Related
ARCHITECTURE.md §4.1(3) says this can never fire — that entry is stale; both sides match in current code (see §5)
Evidence
app/services/notification_service.py:1792-1821 (filter learning.streak.at_risk.ten.*) · publisher micro-learning-service-v2/app/services/streak_reminder_scheduler.py:183 with root micro-learning-service-v2/app/core/config.py:77 · scheduled from micro-learning-service-v2/app/main.py:98-99

Leaderboard movement live

employee

When a learner's standing changes after points are awarded, they can get a push about it — but only if they actually have an active assignment, so people with nothing to do are not pestered, and at most one leaderboard push per person per day. Leaderboard mechanics change fast and the pushes deliberately do not; in-app rows still land regardless of the cap.

Entry points
NATS learning.leaderboard.> (durable notification_worker_leaderboard)
Touches
notifications, Redis daily cap + active-assignment cache, assignment-service internal API, FCM
Related
cap LEADERBOARD_DAILY_CAP_PER_USER=1 (app/core/config.py:107); known ordering defect bug-hunt-reports/notification-worker.md #9
Evidence
app/services/notification_service.py:2296-2320 · publisher micro-learning-service-v2/app/services/leaderboard_diff_service.py:662, triggered from micro-learning-service-v2/app/routes/progress_mobile_routes.py:1254-1257 · gate app/services/assignment_client.py:63

Feed announcements live

employee

An admin posts an announcement to the social feed and everyone in the tenant with a registered device gets it as a push plus an inbox row; single-target feed events go to one person. Each recipient is checked against their own feed-notification preference before the push.

Entry points
NATS training.notifications.ten.*.feed.> (durable notification_worker_feed)
Touches
notifications, user_devices (batched tenant lookup), FCM
Related
known defect: a large-tenant fan-out can outrun the 60s ack window and re-push (bug-hunt-reports/notification-worker.md #6)
Evidence
app/services/notification_service.py:2322-2342 · publisher feed-service/app/events/nats_publisher.py:133 called from feed-service/app/routes/admin_routes.py:121

Quiet hours and daily caps live

employee

A learner's notification settings live in users-auth, and this service asks for them (cached two minutes) before every push: a global push opt-out, a per-category opt-out, quiet hours with a timezone, and a per-day ceiling. Nothing is thrown away when a rule blocks a push — the message is either queued for the end of the quiet window or marked capped, and the inbox row always exists. Escalations are the single intentional exception: they override quiet hours and the cap.

Entry points
Applied inside every consumer; no route of its own
Touches
users-auth /v1/admin/{tenant}/users/{user}/notification/preferences (H25), Redis cap counters, deferred_notifications
Related
caps REMINDER_DAILY_CAP_PER_USER=5, streak window 18–22 local (app/core/config.py:70,95-96)
Evidence
app/services/preferences_service.py:87,160-249 · cap reservation app/services/notification_service.py:876-917 · quiet-window end app/services/notification_service.py:958

Internal (other services)

Which people can be reached by push live

internal

When an admin picks assignees, the UI shows a little indicator next to people who have no phone registered — because assigning them means they will never be nudged. assignment-service and the users-auth back-office ask this service, in one call for up to 1000 people at a time, which of them have at least one active device.

Entry points
POST /v1/internal/devices/batch-active
Touches
user_devices (index idx_user_devices_lookup)
Related
HTTP edges H18 and H21 (ARCHITECTURE.md §3.2)
Evidence
app/routes/api.py:747-749 · callers assignment-service/app/services/notification_worker_service.py:42 and users-auth-service/app/clients/backoffice_clients.py:145

GDPR export slice live

internal

When someone exercises their right of access, users-auth assembles a bundle by asking every service that holds their data for its slice. This service returns the person's notification history and registered devices, scoped by tenant and user. A person with no rows here answers 200 with empty lists — never 404, which would wrongly read as "no such person" — and a partial slice fails loudly rather than being passed off as complete.

Entry points
GET /v1/internal/export/{user_id}
Touches
notifications, user_devices
Related
HTTP edge H23, GDPR fan-out CronJob (ARCHITECTURE.md §3.2)
Evidence
app/routes/api.py:751-753, handler app/routes/api.py:116-168 · caller users-auth-service/app/services/gdpr_export.py:149,156

Background

Deferred delivery retry live

background

A push suppressed because the person was asleep is not lost: it is parked with the time its quiet window ends, and this loop wakes every five minutes, claims whatever is now due, pushes it, and marks the original notification delivered. A row that keeps failing is retried up to five times and then dropped so the queue cannot grow forever.

Entry points
Background task deferred_worker.run
Touches
deferred_notifications, notifications, FCM
Related
known defect: every deferred push is re-rendered as an assignment reminder regardless of its real type (bug-hunt-reports/notification-worker.md #7)
Evidence
started at app/services/notification_service.py:2399-2401 · loop app/services/deferred_delivery_worker.py:31-48

Consumer drain loops live

background

Nine independent loops — one per subscription — pull batches off JetStream and hand them to their handler. Each subscription is created independently, so a broken one cannot take the others down with it, and each drains in its own task, so a slow feed fan-out no longer delays unrelated reminders. When all nine loops are running the process is delivering; when they exit the process shuts down.

Entry points
Background task _drain_loop (one per subscription)
Touches
All 9 MICRO_LEARNING durables
Related
independent-subscribe hardening (to-be-reviewed/notification-worker.md:16-19); per-task drain replaced the ~45s serial cycle
Evidence
app/services/notification_service.py:2423-2458 · isolation wrapper app/services/notification_service.py:2371-2376

Chat-message push dead

background

The service's original purpose, per spec/notification-worker.md, was pushing chat messages to offline users over AWS SNS. That product never shipped: the consumer class still exists but nothing constructs it, no service in the repo publishes a chat subject, and there is no chat service (ARCHITECTURE.md:44 lists oper-chat-service/ as an empty shell). Kept here so the entry point is accounted for, not because it runs.

Entry points
NATS chat.ten.*.channel.*.message.created on stream CHAT_MESSAGES, durable notification_worker
Touches
Nothing — never instantiated
Related
to-be-reviewed/notification-worker.md N-A12, N8
Evidence
app/services/nats_service.py:37-54 · only repo mentions of NATSService are its own definition and docs/ARCHITECTURE.md:206 · no chat. publisher exists in any service

4. API reference

All 15 routes registered by setup_routes (app/routes/api.py:725-777). "Bearer" = a users-auth JWT validated against JWKS; "X-Oper-Key" = the shared internal key. The admin web app and the mobile app are not in this repo, so a route whose only named caller is a frontend doc is cited to that doc.

MethodPathAuthFeatureCallersVerdict
GET/healthznoneLivenessk8s livenessProbe k8s/deployment.yaml:77-80live
GET/readyznoneReadinessk8s readinessProbe k8s/deployment.yaml:85-88live
POST/v1/devices/registerBearer (body user/tenant must match token)Device registrationMobile app (off-repo); global-configs/management_scripts/register_device_keep.py:132live
POST/v1/devices/unregisterBearer (scoped to caller)Device registrationNone in repo. Searched every .py/.md/.dart/.ts for the path: only tests/ and scripts/smoke_test.py:240. Mobile app is off-repo; no FE doc names itsuspect
POST/v1/internal/devices/batch-activeX-Oper-KeyPush reachabilityassignment-service/app/services/notification_worker_service.py:42; users-auth-service/app/clients/backoffice_clients.py:145 (H18, H21)live
GET/v1/internal/export/{user_id}X-Oper-Key + X-Tenant-IdGDPR export sliceusers-auth-service/app/services/gdpr_export.py:149,156 (H23)live
GET/v1/notifications/historyBearerNotification inboxMobile app — docs/flutter_notifications_guide.md:20-21live
POST/v1/notifications/read-allBearerNotification inboxMobile app 5s dwell timer — docs/flutter_notifications_guide.md:23,124live
POST/v1/notifications/{notification_id}/readBearerNotification inboxNone. The mobile guide states it "exists on the backend for completeness but is not used by the app today" — docs/flutter_notifications_guide.md:117-118suspect
POST/v1/notifications/{notification_id}/openBearerNotification inboxMobile app card tap — docs/flutter_notifications_guide.md:22,100live
GET/v1/admin/notificationsBearer + admin.panelAdmin inboxAdmin web — docs/fe-admin-notification-settings.md:174live
GET/v1/admin/notifications/preferencesBearer + admin.panelAdmin settingsAdmin web — docs/fe-admin-notification-settings.md:21,123live
PUT/v1/admin/notifications/preferencesBearer + admin.panelAdmin settingsAdmin web — docs/fe-admin-notification-settings.md:47,123live
POST/v1/admin/notifications/read-allBearer + admin.panelAdmin inboxAdmin web — docs/fe-admin-notification-settings.md:175live
POST/v1/admin/notifications/{notification_id}/readBearer + admin.panelAdmin inboxAdmin web — docs/fe-admin-notification-settings.md:176live

5. Async contracts

Consumes

Ten distinct subjects across 11 scanned registrations. For each one the publisher's real subject was compared token-for-token against the consumer's filter, because a token-count mismatch is silent: the consumer is created, reports healthy, and matches nothing. That is exactly the bug that killed reminders and streaks (to-be-reviewed/README.md #1); the fix is _to_filter_subject (app/services/notification_service.py:99-113), which uses a configured subject verbatim when it already contains a wildcard instead of appending .>. All nine live filters now match their publisher's token count.

Subject (filter)StreamDurablePublished byFeatureVerdict
training.notifications.ten.*.assignment.> MICRO_LEARNINGnotification_worker assignment-service — app/core/nats_client.py:286 emits training.notifications.ten.{t}.assignment.{created|cancelled|completed} (6 tokens; filter's > covers them) Training notificationslive
training.assignments.reminders.ten.* MICRO_LEARNINGnotification_worker_reminders_v2 assignment-service — app/services/reminder_scheduler.py:496, app/services/manual_reminder_service.py:204 emit 5 tokens; filter is 5 tokens Reminders / overduelive
training.assignments.escalations.ten.* MICRO_LEARNINGnotification_worker_escalations assignment-service — app/services/reminder_scheduler.py:529, 5 tokens; filter is 5 tokens Escalation nudgelive
learning.streak.at_risk.ten.* MICRO_LEARNINGnotification_worker_streak_v2 micro-learning-service-v2 — app/services/streak_reminder_scheduler.py:183 builds f"{subject_root}.ten.{tenant_id}" with root learning.streak.at_risk (app/core/config.py:77) = 5 tokens; filter is 5 tokens. ARCHITECTURE.md §4.1(3) is stale — the line it cites (streak_reminder_scheduler.py:190) is the payload's event_type field, not the subject Streak-at-risklive
learning.leaderboard.> MICRO_LEARNINGnotification_worker_leaderboard micro-learning-service-v2 — app/services/leaderboard_diff_service.py:662 emits learning.leaderboard.{type}.ten.{t}; the filter's > matches any depth Leaderboardlive
training.notifications.ten.*.feed.> MICRO_LEARNINGnotification_worker_feed feed-service — app/events/nats_publisher.py:133 emits …ten.{t}.feed.{category}.{subtype} (7 tokens) from app/routes/admin_routes.py:121 Feed announcementslive
training.notifications.ten.*.admin.> MICRO_LEARNING (settings.reminder_stream, app/services/admin_notification_service.py:585) notification_worker_admin assignment-service app/services/admin_notifications.py:59; authoring-service-v2 app/publishers/admin_notifications.py:221; users-auth-service app/routes/mobile_profile.py:216 and app/services/gdpr_export.py:347 Admin envelopeslive
training.assignments.ten.*.assignment.expansion_completed MICRO_LEARNINGnotification_worker_admin_exp_ok assignment-service — app/core/nats_client.py:228, called from app/workers/expansion_worker.py:495; 6 tokens both sides Rollout finished alertlive
training.assignments.ten.*.assignment.expansion_failed MICRO_LEARNINGnotification_worker_admin_exp_fail assignment-service — app/core/nats_client.py:248, called from app/workers/expansion_worker.py:391,525; 6 tokens both sides Rollout failed alertlive
chat.ten.*.channel.*.message.created CHAT_MESSAGESnotification_worker Nobody. No service in the repo publishes any chat.* subject; the only occurrences of this string are app/services/nats_service.py:47 and the stream definition k8s/nats-screams-cm.yaml:11 Chat-message push (never shipped)dead

The scan JSON lists this subject twice. The second entry (app/services/admin_notification_service.py:582) is a scanner mis-attribution: line 582 is the generic _pull_subscribe helper whose subject= is a parameter, and the scanner resolves an unbound subject name from its service-wide constant table (tools/feature-docs/scan.py:298-299), picking up the literal from nats_service.py:47. The helper's real subjects are the three admin settings values at admin_notification_service.py:596-612, each already listed above.

Ops note, not a subject mismatch: the streak and leaderboard consumers bind to MICRO_LEARNING, but this service's own copy of the stream definition omits learning.> and user.notification.> (k8s/nats-screams-cm.yaml:31). The canonical definitions do include them (micro-learning-service-v2/nats/streams/training-modules.json:10, assignment-service/nats/streams/micro-learning.json:10) and this service's init scripts are never executed (§8), so the live stream is correct — known defect ARCHITECTURE.md §4.1(4).

Publishes

SubjectConsumed byFeatureVerdict
auth.admin.alert (stream AUTH) otp-worker — subscribed at users-auth-service/app/otp_worker/config.py:39, dispatched at users-auth-service/app/otp_worker/processor.py:161-162 → Mailgun Admin email mirror for critical_training and overdue_digest live

Background jobs

JobScheduleWhat it doesVerdict
deferred_worker.run (app/services/deferred_delivery_worker.py:31) Every 300s while the process lives (DEFERRED_DELIVERY_POLL_SECONDS) Claims deferred_notifications rows whose quiet window has ended, pushes them, marks the original row delivered; gives up after 5 attempts live — started at app/services/notification_service.py:2401
_drain_loop (app/services/notification_service.py:2423) Continuous; one task per subscription, 5s fetch window Pulls batches from each of the 9 durables and runs its handler; a failed subscription is skipped instead of killing the others live — 9 tasks created at app/services/notification_service.py:2455-2458

No CronJob exists for this service; both jobs are in-process tasks of the single notification-worker Deployment. A SIGTERM rollout skips their graceful stop — known defect bug-hunt-reports/notification-worker.md #10.

6. Data it owns

Postgres logical DB notifications; notification-worker is the only writer (ARCHITECTURE.md §3.4). Four live tables after alembic/versions/drop_unused_tables.py removed the chat-era channel_members and notification_history.

TableWhat it holdsWritten by
user_devices One row per phone: tenant, user, FCM token (globally unique), platform, device name, active flag. Deactivated when Firebase reports the token dead. notification-worker only — app/routes/api.py:285-345 (register/unregister), app/services/notification_service.py:292-304 (auto-disable). Created in alembic/versions/initial_schema.py:37.
notifications The inbox. Title, description, deep link, type, tags, data blob, audience (learner or admin), plus the tracking columns status, delivered_at, read_at, opened_at and the idempotency event_id. notification-worker only — consumers insert, the HTTP inbox routes update read_at/opened_at. alembic/versions/add_notifications_table.py:22, extended by 2026_05_22_add_reminder_delivery.py:26-56 and 2026_07_03_admin_centre.py:26-54.
deferred_notifications The quiet-hours parking lot: which notification to re-push, for whom, when, and how many attempts have failed. FK cascade from notifications. notification-worker only — enqueued by the consumers, drained by the deferred worker. alembic/versions/2026_05_22_add_reminder_delivery.py:58, uniqueness added in 2026_07_07_unique_deferred_notification.py:36.
admin_notification_preferences Per-admin category switches and the email-enabled flag, keyed (tenant_id, user_id). notification-worker only — PUT /v1/admin/notifications/preferences. alembic/versions/2026_08_01_admin_notification_preferences.py:27, email column in 2026_08_01_admin_email_prefs.py:26.

No other service writes into this database, and this service writes into no other database. Everything it needs from elsewhere it fetches over HTTP: learner notification preferences and the tenant's admin list from users-auth (the notification_preferences table lives in users_authusers-auth-service/app/services/gdpr_export.py:110), and the active-assignment check from assignment-service. Redis db 1 holds only derived state: daily-cap counters and short-TTL caches.

7. Dependencies

flowchart LR
  MOB["Employee app (off-repo)"] --> NW["notification-worker"]
  ADMW["Admin web (off-repo)"] --> NW
  AS["assignment-service (H18)"] --> NW
  UA["users-auth-service (H21, H23)"] --> NW
  ML["micro-learning-service-v2"] -. NATS .-> NW
  FS["feed-service"] -. NATS .-> NW
  AV2["authoring-service-v2"] -. NATS .-> NW
  NW --> UAO["users-auth: admins, preferences (H25)"]
  NW --> ASO["assignment-service: active assignments (H26)"]
  NW --> FCM["Firebase FCM"]
  NW -. "auth.admin.alert" .-> OTP["otp-worker: Mailgun email"]
  NW --> PG["Postgres: notifications"]
  NW --> RD["Redis: db 1"]

8. Dead-code verdicts

Every entry point with no in-repo caller. Deleting is a separate decision — see the hub roll-up.

Entry pointKindVerdictEvidence
chat.ten.*.channel.*.message.created on stream CHAT_MESSAGES, durable notification_worker NATS consumer dead Both halves. (1) Never started: the registration lives in NATSService.subscribe_to_channel_messages (app/services/nats_service.py:37-54); a repo-wide search for NATSService/nats_service returns only that class definition and a mention in docs/ARCHITECTURE.md:206main.py:20 and NotificationWorker.run() (app/services/notification_service.py:2344-2465) construct NotificationWorker and AdminNotificationProcessor only. (2) No publisher: no service emits any chat.* subject; the string exists only here and in k8s/nats-screams-cm.yaml:11. Corroborated by to-be-reviewed/notification-worker.md N-A12.
POST /v1/devices/unregister HTTP route suspect Searched the whole repo (.py, .md, .dart, .ts) for the path: matches are only app/routes/api.py:746, tests/test_notification_endpoints.py, scripts/smoke_test.py:240 and the service's own docs/ARCHITECTURE.md:627. Tests and smoke scripts do not count as callers. The mobile app that would call it on sign-out is not in this repo, and no fe-*/Flutter integration doc names it. Resolve by checking the mobile client or the ingress access log.
POST /v1/notifications/{notification_id}/read HTTP route suspect The mobile integration guide states it outright: "The /v1/notifications/{id}/read endpoint exists on the backend for completeness but is not used by the app today" (docs/flutter_notifications_guide.md:117-118). The current UX marks read via /open on tap or /read-all on dwell. Not dead: a mobile release could start calling it without any backend change.
NATSService class (app/services/nats_service.py:11-78) Module / class dead Unreachable in principle: nothing imports or instantiates it (only self-reference + docs/ARCHITECTURE.md:206), it duplicates the connection logic NotificationWorker.connect_to_nats already owns (app/services/notification_service.py:148-162), and its single method targets a stream no publisher feeds. Its own drain loop is also unreferenced by the scan's background-task list.
CHAT_MESSAGES stream definition (k8s/nats-screams-cm.yaml:8-26) Config / stream dead Defines a stream for the dead chat subject, and nothing applies it: the ConfigMap is mounted as a volume but no initContainer runs it — k8s/deployment.yaml:18-31 runs only alembic upgrade head. Same for k8s/nats-init-script.yaml. Documented as an operational hazard (an operator running it by hand could shrink the shared MICRO_LEARNING subject list, ARCHITECTURE.md §4.1(4)); see to-be-reviewed/notification-worker.md N-A6, N-A12.
spec/notification-worker.md (AWS SNS / chat / presence design) Doc dead Describes SNS push for offline chat users and a TRAINING_NOTIFICATIONS stream; the live implementation is FCM on MICRO_LEARNING. Flagged as stale in to-be-reviewed/notification-worker.md N8. Listed for completeness — documentation, not an entry point.

Everything else in the scan is live: 13 of 15 routes have a named caller or probe, 9 of 10 consumed subjects have a confirmed publisher with a matching token count, the single published subject has a confirmed consumer, and both background tasks are started from NotificationWorker.run(). Open correctness defects that are not dead code — duplicate pushes on JetStream redelivery, deferred pushes mislabelled as reminders, the feed fan-out ack window, the internal API key in a tracked ConfigMap — are tracked in bug-hunt-reports/notification-worker.md and to-be-reviewed/notification-worker.md, not restated here.

9. Sources