Notification infrastructure integration¶
Temporary planning document; planning only. Chris added the existing, current and planned notification infrastructure to this programme's scope on 3 October 2026. This is not authorisation to implement notifications, to send messages to project users or to change the notification PRs. The notification stack belongs to a separate programme: a local conversation ("SyRF Notifications and Study Issues") authorised its implementation on 2 October and delegated it to the Juniper thread "SyRF: Inspect notification and email infrastructure". This plan consumes its contracts and proposes amendments; it does not take it over.
Evidence: a read-only sub-agent inspected the eight PR worktrees (each confirmed clean and at its
PR head) and the provenance transcript (targeted excerpts only). PR states were re-read live with
gh on 3 October at about 04:52 BST. File references below are in the top-of-stack worktree
pr/pr3947.checked-pdf-proposals-and-explicit-administrator-a-r6bdl2 unless stated. The
adversarial reviews re-checked the #3944 behaviours in its own worktree.
1. Verified baseline¶
1.1 What is merged on main (78c6d097d, unchanged at 2949ca3a7)¶
- Email transport: SES by default, SMTP to Mailpit outside production,
SendSimpleEmail, development routing (api/Models/ApiEmailTransportRegistration.cs:23-44,api/Models/AwsEmailService.cs:190-192, 293-307). - The SignalR hub with per-user groups, plus live, unsaved banners: surplus warning,
ActivityClaimRevoked, disconnection/idle (web/core/services/signal-r/signal-r.service.ts:147-154). - A dead legacy
Notificationclass (InvestigatorAggregate/Notification.cs), unused. - The Identity service's recovery-email outbox, an earlier pattern for leased, no-retry sends.
1.2 What exists only in the open stack (all open, stacked, flags off, none merged)¶
States as read on 3 October at 15:18 BST, except #3965, re-read at 20:35 BST.
| PR | Head | Adds | State |
|---|---|---|---|
#3932 (base main) |
2ddd96fb0 |
InboxNotification aggregate (pmInboxNotification), inbox API and /inbox UI, export-completed capture in the terminal export transaction, change-stream InboxChanged hint |
Conflicting with main (generated files only) |
| #3938 | 180f0d1bb |
Import-completed (recipient fixed at upload admission) and project-invitation capture; typed ISourceInboxCapture |
Mergeable in stack |
| #3941 | eb72c886d |
reviewAccessGranted and workloadChanged from before/after snapshots of effective Review grants and workload |
Mergeable in stack; wraps ProjectController.UpdateProject, which #3964 (merged 3 October) also edits |
| #3942 | 75f89b33d |
Opt-in email preferences per category (default off), immediate delivery with a per-item ledger, leases and ambiguous-send handling | Mergeable in stack |
| #3943 | d75fd81b9 |
Durable daily digests (IANA timezone, DST handling, at most 100 unread items frozen when claimed) | Mergeable in stack |
| #3944 | 59ab32e7b |
StudyConversation: reconciler questions to selected candidates and replies (studyAttention flag) |
Mergeable in stack |
| #3945 (on #3944's branch) | 25b364da3 |
StudyIssue reports with open/resolved/reopened status and administrator resolution (studyAttention); writes Study bibliographic fields |
Mergeable in stack |
| #3947 (on #3945's branch) | ae3c6749d |
Checked PDF proposals (quarantined, scanned, administrator-approved), using the study-issue kind; writes the Study PDF path; adds native tools to the API image | Mergeable in stack |
| #3965 (on #3947's branch) | 986b1cdc2 |
Chris's Q-10 changes: reconciliationConversations flag depending on notificationInbox; one-to-one threads; completed non-reconciliation sessions only; a content-free "questioned" marker; read-only context; all-or-nothing thread creation; stable reviewer labels; typed reply refusals; a reconciler who reviewed the study in any stage is refused |
Open and ready for review; ten commits pushed (20:35 BST); its local e2e journey passed (19:30 BST reading); waits for the stack (D1-09) |
Checks and review: E2E jobs were skipped on every stack PR (no run:e2e-* label), so the stack's
new E2E specs have never run in CI. Product CI runs only while a slice is temporarily retargeted to
main. The latest-head Claude reviews were short reviews of main-merge deltas; #3932 has no human
approval. Issue #3950 lists 13 follow-ups; it does not track retention, fan-out limits, unsubscribe,
the email flag dependency, a delivery halt or per-project admission (those are untracked).
Live on merge, even with flags off: unflagged preference endpoints; five web routes guarded only by sign-in; isolated reads and a join-response fix on project endpoints; three hosted services (the change-stream watcher, the email worker and the digest worker) in every API replica; native PDF tools in the API image (#3947).
1.3 Contract facts the review programme must respect¶
- The saved inbox row is the durable obligation. It is written in the same Mongo transaction as
the source change (snapshot read, majority write);
SourceInboxCapture.cs:58-60refuses writes outside a source transaction. Under C15 v2 that remains the inline mode; events with no such transaction use recorded fan-out (a durable intent expanded by a leased worker). There is no second notification store, and in-memory outboxes stay forbidden. - Deduplication uses the unique index (RecipientId, Kind, SourceId) with an insert-only upsert
(
SourceInboxCapture.cs:62-71,InboxNotificationRepository.cs:38-39), plus deterministic SHA-256 identifiers for explicit-request workflows (StudyConversation.cs:20-25,StudyIssue.cs:24-27). A second event with the same SourceId is silently dropped, so SourceId must identify the event occurrence deterministically.reviewAccessGrantedusesGuid.NewGuid()and relies on snapshot comparison; copied into a resumable batch it would duplicate notices. - A Kind is also an email category key, and categories are validated as an exact set. Adding a
kind needs the constant on
InboxNotification, aNotificationDetailResolverbranch with an access class, an entry inNotificationEmailPreferences.CategoryNames, the Angular preferences list and an inbox action label. Because the set is exact, every new kind breaks preference saves (including opt-out) whenever browser and API disagree; without the resolver branch the inbox shows "Related item unavailable" and the email is silently suppressed. - Payloads are generic: titles and links only. Protected detail is shaped at read or send time by the resolver, with fresh authority checks and no application-admin shortcut for review items.
- Recipients are individuals expanded at write time. Revocation hides details but keeps the item in history and in the unread count. People granted access after an event get no item for it.
- Read state never resolves a workflow. Workflow state belongs in the feature aggregate.
- Email: opt-in per category, immediate or daily, one ledger row per item, ambiguous sends never retried, no unsubscribe link yet. The workers run only in the API; the Project Management service's email service throws, and that service doesn't receive the notification flags.
- Flags:
notificationInbox,notificationEmailandstudyAttention(which requiresnotificationInbox) all default off, are environment-wide and reach only the API and web.studyAttentionadmits conversations, study reports and authorised decisions together; #3965 moves conversations to their ownreconciliationConversationsflag.notificationEmaildoesn't declare its dependency onnotificationInbox. The workers run regardless of flags, and accepted immediate and digest mail keeps sending afternotificationEmailis turned off. In staging and preview any SyRF administrator can override these flags at runtime. - No retention, TTL, archive or delete exists for any notification collection.
- #3944 reads legacy sessions. It finds candidates and authorises replies through
Study.ExtractionInfo.Sessionsand stores legacy session IDs (StudyConversationAccess.cs:46, 52;StudyConversation.cs:28), so it would be inert for canonical sessions, and its references would dangle after adoption. - Five inbox writers use different identity rules (
SourceInboxCapture,DataExportJobRepository,StudyConversationRepository,StudyIssueRepository,CheckedPdfProposalRepository); the study-issue and PDF writers useInsertOnerather than an upsert. - Inbox reads are gated by the capture flag: turning
notificationInboxoff returns 404 for every inbox read, which breaks links already emailed. - Access is checked against the captured
StageId, which fails for a study × form task reachable through several stages (RE4). - Extension points are per-kind code in shared files (resolver switch, typed fields on the row and DTO, a browser label chain whose fallback labels any new kind "Open conversation"), so every lane adding a kind would edit the same notification-owned files. C15 v2's kind registry replaces this.
2. #3944 reconciliation questions versus confirmed decisions¶
3944 is real runtime code (aggregate, repository, fresh authorization, API, Angular UI,¶
transactional notifications, tests), not a proposal. It implements a reconciler → selected candidate question-and-reply channel. Measured against the owner ledger:
| Behaviour in #3944 (as inspected) | Confirmed decision affected | Integration finding |
|---|---|---|
Every participant sees every post, and replies notify all other selected reviewers (StudyConversationRepository.cs:64-70; study-conversation.component.html:12-25) |
VS1 candidate isolation ("other candidates' individual answers remain hidden") | A reviewer's explanation of their answer reaches the other candidates. Conflict. Proposed: reviewer-private (one-to-one) threads. |
Incomplete sessions are eligible candidates (StudyConversationAccess.cs:52), and the reviewer's context link opens the editable review route (StudyConversationsController.cs, ContextPath to /review/) |
PV1/VS2 principle: record what a contribution was exposed to | A candidate can be questioned before completing, or after a reopen, and then change their answers, with nothing recorded. VS2 itself concerns accepted answers, so this is an independence and exposure risk extending that principle. Proposed: an exposure marker on any questioned session plus a read-only context link. |
Any Reconcile holder can start a thread; only their own session is excluded from the candidates (StudyConversationAccess.cs:23-35, 49-56) |
"A reconciler is never offered a study they reviewed" (R4a acceptance; the ledger allows audited self-review only for query review, QY4; project-level exceptions are Q-36) | Gap. Proposed: a reconciler-eligibility check before a thread can start. |
Thread keyed by stage; owned by the creating reconciler (StudyConversation.cs:8-29; access :37) |
RE4 (one task per study × form across stages); RA4 (release and reacquire) | A shared form reconciled through another stage, or a reconciler taking over after release, can't see the discussion. Gap. Proposed: bind threads to the reconciliation task identity once R4a exists; allow authorised handover. |
| Generic labels ("Reviewer N") whose numbering differs between picker and thread (#3950 item 10) | BL1 (stage-owned, consistent identity blinding) | Gap. Proposed: one stage-owned alias source shared by candidate cards, threads, issues and history. |
| No reconciliation-state check; usable after gold exists | QY1–QY9 (audited, per-concern query lifecycle) | Risk of an informal query channel that bypasses the audited lifecycle. Gap. Proposed: threads only while a reconciliation task is open; post-gold challenges go to queries (R4b). |
| Additional reviewers become candidates once they start a session | RA5 (independent additional review) | An extra reviewer could be questioned before completing. Conflict. Proposed: exclude requested additional reviewers until they return their assessment. |
Decision (Q-10, Chris, 3 October): make these changes. #3965 implements them in ten commits
(head 986b1cdc2, pushed and ready for review at 20:35 BST on 3 October; its local e2e journey passed
at the 19:30 BST reading): conversations move to their own reconciliationConversations flag, which declares
its dependency on notificationInbox, and studyAttention's description no longer mentions
conversations; threads are one-to-one between the reconciler and one candidate and are created
all-or-nothing; only completed non-reconciliation sessions are candidates, which also keeps requested
additional reviewers (RA5) out until they return their review (answering NS's Q-N9; recorded in the
register, no question); a content-free "questioned" marker is shown for every reconciler; reviewer
labels are stable; the reviewer's context link is read-only; refused replies are typed; a reconciler
who reviewed the study in any stage is refused. Still outstanding:
- Cross-stage reconciler refusal (legacy part done). The tenth commit (
986b1cdc2) refuses a reconciler with any non-reconciliation annotation session on the study, in any stage and any status. That is stricter than the legacy rule proposed here (refuse a reconciler with any non-reconciliation session on the study in any stage whose questions overlap). Still to come, from R4a: refuse any session on that study × form (AC-R4a-21). - Canonical scopes. Conversations read legacy sessions, so they refuse canonical scopes through
R0's ownership record until R4a rebinds them to
ReconciliationTask; L6 ownsStudyConversationfrom R4a. - Exposure. The "questioned in reconciliation" record feeds C3's exposure states, R5c and R6 manifests (AC-R4a-35, AC-R5c-07).
Merge route (D1-09, approved by Chris on 3 October): restack #3965 onto #3944 and merge it straight after #3944, before #3945 and #3947; if the stack owner prefers to keep it on #3947, it lands in the same merge train before any environment enables conversations. Task identity, handover, stable aliases and open-task-only threads remain R4a follow-ups.
3. Review-workflow notification catalogue¶
None of these triggers exists yet. Each is delivered by the lane and release shown through C15 v2 (contracts). Feature-owned queues (§4.1) are the source of truth; these notices only add delivery. Every kind registers its email category, capture mode, occurrence key and scope.
Email categories (PROPOSAL; the eight existing kinds keep their own keys):
| Email category | Kinds |
|---|---|
| Access and workload (existing two keys kept) | Review access; Reconcile access (activity field); workload, including canonical workload changes from the form target and the AL1 plan |
| Work assigned to me | Assignment created, expiry warning, expired, released; additional review requested, withdrawn, expired; concern needs applicability review |
| Results of my requests | Additional review returned; concern outcome (QY6); addressed by update (QY8); change approved or declined (LC1) |
| Approvals I need to give | LC1 change awaiting approval; outcome-migration manifest awaiting approval |
| Changes affecting my reviews | Form version published (one kind); FV4 revision; profile version published; outdated annotations caused by someone else; task held; inputs changed |
| Project changes | Adoption cutover; pilot admitted, removed or made read-only; publication applied or stalled (to the publishing admin) |
| Existing kinds | Exports; imports; invitations (no notification email: the invitation email already goes, NS-25); conversations (one-to-one; legacy until R4a); study issues |
| Event | Recipients | Capture mode | Occurrence key | Scope | Default channel (PROPOSAL) |
Release |
|---|---|---|---|---|---|---|
| Form version published (re-answer required, autoUpdate applied or no action), one kind with the per-recipient effect worked out at read time | Owners of affected current sessions, once per recipient per publication | Recorded fan-out from the frozen manifest | Publish operation ID | Canonical | In-form Needs updating; in-app notice; email per preference | R2c |
| Publication applied, completed or stalled | The publishing admin | Inline | Publish operation ID and phase | Canonical | In-app | R2c |
| Older notices for the same form superseded by a newer publication | Earlier recipients | Inline (state change on existing rows) | Form ID and publication sequence | Canonical | Marked superseded | R2c |
| Session contains outdated annotations caused by someone else (support on-behalf-of write, adoption remap or shared-gold revision; Q-20 narrowed; a publication writes no revision under D2-01, so it never causes this flag and reaches reviewers through the "form version published" kind) | Session owner, one per cause | Inline for one session; recorded fan-out for remaps | Cause ID (command, remap or gold snapshot) | Canonical | In-app flag on the session; optional notice | R2d |
| Requirement revised so re-answering is no longer needed (FV4) | Owners of sessions asked to re-answer | Recorded fan-out | Policy revision | Canonical | In-app | R2d |
| New screening-profile version published with action needed (Q-26) | Reviewers with decisions under prior versions, if the treatment needs action | Recorded fan-out | Profile publish operation ID | Canonical | In-app | R3b |
| Completed-stage change awaiting approval (LC1) | Admins holding the approval capability for every affected stage | Inline up to the cap, otherwise recorded fan-out | (Change request ID, opened) | Canonical | "Changes awaiting approval" queue; badge; in-app; resolved for all once one admin decides (D3-23) | R3c |
| Change approved or declined (LC1) | Requester | Inline | (Change request ID, decision) | Canonical | In-app | R3c |
| Drafts or corrections blocking automatic completion (LC1, reviewer side) | Owners of the blocking drafts or corrections | Recorded fan-out | (Stage ID, readiness evaluation) | Canonical | In-app | R3c |
| Stage completed or reopened | Stage admins | Inline | (Stage ID, status-history entry) | Canonical | In-app | R3c |
| Reconciliation assignment created, expiry warning, expired, released (RA2–RA4), one occurrence each | Assignee; on release also the previous holder | Inline; warnings and expiry through a scheduler marker on the assignment | (Assignment ID, transition, version) | Canonical | "Assigned reconciliation work" queue; in-app; email per preference | R4a |
| Additional review requested, withdrawn or expired (RA5) | The requested independent reviewer | Inline; expiry through a scheduler marker | (Request ID, transition) | Canonical | "Requested reviews" queue; in-app | R4a |
| Additional review returned | The requesting reconciler | Inline | (Request ID, returned) | Canonical | In-app | R4a |
| Reconciliation clarification question or reply (#3944, #3965) | The thread's two participants | Inline | Thread and post IDs (deterministic) | Legacy until R4a, then canonical through the task | Existing kinds | R4a alignment |
| Item held because a publication made it incompatible | Reconcilers with access to the task | Inline | (Task ID, held episode) | Canonical | Pool banner (SignalR); in-app | R4a |
| Corrected input needs a re-check ("inputs changed · re-check") | The reconciler or task holder | Inline | (Task ID, drift episode) | Canonical | Task state; in-app | R4a/R4b |
| Query raised on an accepted answer | Holders of the query-review capability for the stages reaching the task | Inline up to the cap, otherwise recorded fan-out with a capability selector | (Query ID, concern ID) | Canonical | Query queue; in-app | R4b |
| Concern resolved, rejected or "addressed by update" (QY6, QY8) | Each raiser individually | Inline | (Concern ID, outcome version) | Canonical | "My concerns and outcomes"; in-app; email per preference | R4b |
| Concern needs current-applicability review (QY9) | Assigned query reviewer | Inline | (Concern ID, applicability check) | Canonical | Query queue; in-app | R4b |
| Review or Reconcile grant changes | Recipient | Inline, capped (bulk group edits use recorded fan-out) | (Mutation command, stage, activity) | Both | #3941's access snapshot, extended with Reconcile grants and the active decision path | R1c, R4a |
| Adoption cutover; pilot admitted, removed or made read-only (U28) | Project members | Recorded fan-out | (Wave or admission version) | Both | In-app | R0/R6 |
| Outcome-migration manifest awaiting approval (O2) | Owner or migration delegate | Inline | (Manifest ID, version) | Canonical | In-app | O2 |
Live banners (publication in progress, claim revoked, idle) use the existing per-user SignalR groups, not C15. Routine claims never notify (RT-25).
4. Integration contract (C15 v2)¶
The contract text is C15 in contracts. For lanes adding a review-workflow kind:
- Register the kind (category, scope, admission flag, inline limit, resolver) and its disclosure
fixtures; the registry test fails otherwise. Never edit the shared resolver switch or add a typed
field to
InboxNotification. - Choose the capture mode per operation: inline inside the domain transaction with bounded recipients (Save/Complete: task holder, requesting reconciler; gold publication and query resolution: raisers), or recorded fan-out keyed by durable identity (publications, adoption, bulk grant changes). Benchmark hot paths with capture on.
- Derive the occurrence key from durable identity; never a random SourceId.
- Expand recipients with current authority when the row is written or the intent is expanded; exclude the actor; per-raiser notices go only to the raiser. Feature queues handle later joiners.
- Shape details through the kind's resolver and the disclosure hook, for HTTP, inbox and both email processors; on any authority miss show "Related item unavailable".
- Live updates: nothing to add; the invalidation worker covers any inbox write (with jittered refetch, §5).
- Email: map the kind to one of the categories in §3; no new ledger, worker, digest, Quartz job, preferences store or transport.
- Flags: new writes behind a default-off flag that declares its dependency on
notificationInbox; capture also requires per-project notification admission. Reading accepted history stays available, still authority-checked. Producers in the Project Management service use C15 v2's capture service (not a host-specific path). - Workflow state stays in the feature aggregate, never in
ReadAtUtc. - Tests: C15-T01 to C15-T11 (acceptance criteria §7.5; the NS review's AC-C15-01 to 09 are C15-T01 to T09, and AC-C15-10 is C15-T11 with AC-ALL-04 (d)): capture rollback, replay and resume idempotency, registry, disclosure per channel and role, admission, version skew, halt, fan-out, coalesced hints, rollback.
- Never send test mail to users. Use Mailpit and the hermetic e2e stack.
4.1 Feature-owned queues (source of truth and fallback)¶
Confirmed obligations never depend on the notification stack being merged or enabled:
| Queue | Carries | Release |
|---|---|---|
| My concerns and outcomes | QY6 per-raiser outcomes; QY8 "addressed by update" | R4b |
| Changes awaiting approval | LC1 admin alert and confirmation | R3c |
| Assigned reconciliation work | RA2–RA4 assignments, expiry and release | R4a |
| Requested reviews | RA5 additional reviews | R4a |
Surfacing without notifications (NS-07, PROPOSAL; placement per D3-07): a global navigation
badge whose counts are computed at read time over the four queues; a cross-project "My work" tab beside
"Pending Projects" on the project index; a project-overview banner for admins with pending LC1 requests.
With every notification flag off, a user reaches an assignment in a project they have not opened from
the project index in one step, and an admin with a pending request sees the badge without opening the
project (AC-R3c-11, AC-R4a-47). Each release's acceptance tests pass with every notification flag off.
5. Dependencies and gates¶
| Need | Depends on | Gate |
|---|---|---|
| Notices for review-workflow events | X-NOTIF: stack steps 1–5 (#3932, #3938, #3941, #3942, #3943) merged with flags off, and C15 v2 landed before the first review-workflow kind | Per release. Without it, features still ship because the queues in §4.1 carry the confirmed obligations |
| Any enablement outside the e2e stack and Mailpit | G-NOTIF: Chris approves each environment and kind family (D3-21); the delivery halt, inbox reads independent of capture and per-project notification admission delivered (E74); no reliance on runtime overrides while #3975 is open | Per environment and kind family |
| Preference saves across versions | Tolerant preferences and the kind→category map, before #3942 merges (or one deploy before the first new category) | Before #3942 merges |
| Enabling conversations | #3965 (ten commits pushed and ready for review; includes the legacy cross-stage reconciler refusal) reviewed and merged after #3944 (D1-09) | Before enabling; F4 |
| R1c group and grant changes | #3941 merged (step 3), aligned to the active decision path with Reconcile grants, with bulk fan-out through recorded fan-out | R1c |
| Canonical sessions in #3944 | Conversations refuse canonical scopes until R4a rebinds them to ReconciliationTask; their aggregates in R6 manifests |
R2a (refusal), R4a, R6 |
| High-volume events (R2c publication, R2d outdated flags) | Recorded fan-out with a recipient threshold; flood controls (mark-all-read, filters, context line, hide unavailable, resolved and superseded state, grouped digests, jittered refetch) | Before R2c |
| Email for new kinds | notificationEmail dependency declared; unsubscribe policy; per-project email mute (D3-24) |
Before any email category is enabled outside Mailpit |
| Stack screens | UI-1 to UI-11: inbox and preferences before testers see them; conversation, issue and PDF screens before production (D3-01) | Per D3-01 |
Merge order (D1-09, approved by Chris on 3 October as recommended;
register §1.13): step 0 the ownership fix #3964 (done: merged on 3 October as 85e6facf7,
D1-01 carried out; it edits
ProjectController.UpdateProject, which #3941 wraps) → #3932 (regenerated generated files on current
main, full checks on the retargeted head, run:e2e-full, human approval) → #3938 → #3941 (rebased on
step 0, with a test that a refused owner-reserved grant captures nothing) → #3942 (with tolerant
preferences) → #3943 → #3944 → #3965 (restacked if the stack owner agrees; otherwise as in §2's
merge route) → #3945 → #3947 last (native PDF tools moved out of
the API image, or the isolation review finished).
Ownership: the notification programme keeps capture, inbox, email and digests. StudyConversation
moves to L6 at R4a; study issues and checked PDFs move to the Study Management and PDF programmes. An
ADR "Saved notification capture and delivery" lands with #3932 or with C15 v2, and the MongoDB reference
lists the new collections.
6. Open items for Chris¶
- Q-10: decided by Chris on 3 October; implemented in #3965 (ten commits pushed, ready for review; includes the legacy cross-stage reconciler refusal; waits for the stack, D1-09).
- Batch D: D3-01 (screen timing), D3-07 ("My work"), D3-21 (enablement), D3-22 (project name in email), D3-23 (auto-resolve), D3-24 (per-project email mute), D3-25 (conversations as audit record). D1-09 (merge order) was decided on 3 October; restacking #3965 onto #3944 still needs the stack owner's agreement.
- Q-20: narrowed to outdated annotations someone else caused (
PROPOSAL). - Untracked items for the notification programme (not in #3950): retention and erasure for
preferences, the delivery ledger, digests, posts and PDF bytes (E32); fan-out limits; unsubscribe; the
notificationEmaildependency; the delivery halt; per-project admission; idle workers; route guards; duplicate invitation email (NS-25).