Skip to main content

Expense Event Delivery Troubleshooting

Summary​

Downstream event is delayed, outbox remains Pending/Failed, or Processed publication has no visible consumer outcome.

Audience​

  • Employees, managers, HR administrators, support engineers, developers, QA engineers, DevOps engineers, and solution architects, as applicable to the symptom.

Cause​

Use the verified possibilities in Likely causes; do not infer an environment-specific cause without evidence.

Resolution​

Use Safe resolution, then complete Validation. Stop and escalate if the required action exceeds approved access.

Symptoms​

Downstream event is delayed, outbox remains Pending/Failed, or Processed publication has no visible consumer outcome.

Scope​

Expense outbox, hosted relay, broker publication, and independent consumer processing.

Likely causes​

  • Relay has not selected backlog
  • Broker publication unavailable
  • Repeated failure with no retry ceiling
  • Processed proves publication but not consumption
  • Consumer is delayed, failed, duplicate, or stale
  • No dead-letter/replay workflow exists

Safe diagnostic steps​

  • Confirm expected event type was created
  • Inspect status/retry metadata without viewing payload
  • Check relay logs/traces and service health
  • If Processed, move investigation to the specific consumer
  • Do not republish, replay, or edit outbox records

Safe resolution​

Allow normal relay/consumer retry when safe; escalate persistent backlog or consumer failure. Recovery/replay requires owner-approved procedures that are not currently documented.

Validation​

Outbox reaches Processed and the intended consumer records its own outcome.

Evidence to collect​

Event type/identity, state, retry count category, occurrence/processed times, correlation, relay and consumer result—no payload.

Escalation​

Expense/messaging owner for backlog/publication; downstream service owner for consumption; operations owner for incident coordination.

Requires confirmation​

Any corrective action beyond the supported resolution, incident ownership, response objectives, and production evidence locations require confirmation.

Source References​

  • microservices/src/expense-service/Infrastructure/Persistence.cs
  • microservices/src/shared-kernel/Messaging/OutboxProcessor.cs
  • microservices/src/expense-service/Messaging/OutboxRelayHostedService.cs

See Also​

Keywords​

  • Expense troubleshooting
  • Event Delivery

Revision Information​

  • Status: Draft
  • Last reviewed: 2026-07-16
  • Review cycle: Quarterly