Skip to main content

Retry and Failure Handling

Summary​

Producer and consumer retries are unbounded in the inspected lifecycle paths. No dead-letter, terminal-failure or operator replay implementation was found.

Audience​

Reliability engineers, service owners, QA, support, and architects.

Reference Content​

Producer behavior​

OutboxProcessor increments RetryCount and stores failure detail after a false publish result or exception. Stores select pending and failed entries again. No maximum retry, backoff by message, terminal status or dead-letter transition was found.

Recruitment's relay catches iteration errors and continues after its fixed poll delay. Employee has no registered relay, so producer retry is not active there.

Consumer behavior​

Payroll, Leave and Notification retry connection establishment after failure. Processing exceptions are negatively acknowledged with requeue. Successful/duplicate/skipped local processing is acknowledged after persistence behavior returns.

Partial-failure windows​

Employee provisioning is synchronous before Recruitment completion. If Employee commits and Recruitment later fails, messaging retry cannot reconcile the split. Notification delivery failure does not roll back Employee creation.

Poison-message handling, alert thresholds, replay authorization and failure ownership require confirmation.

Requires confirmation​

Unless explicitly confirmed above, live delivery, consumer deployment, replay, reconciliation, retention, contract governance, monitoring and production ownership require confirmation.

Source References​

  • microservices/src/shared-kernel/Messaging/OutboxProcessor.cs
  • microservices/src/recruitment-service/Messaging/OutboxRelayHostedService.cs
  • microservices/src/payroll-service/Messaging/PayrollReadModelConsumer.cs
  • microservices/src/leave-service/Messaging/EmployeeProfileConsumer.cs
  • microservices/src/notification-service/Messaging/NotificationEventConsumer.cs

See Also​

Keywords​

  • Onboarding events
  • Offboarding events
  • Integration contracts

Revision Information​

  • Status: Draft
  • Last reviewed: 2026-07-20
  • Review cycle: Quarterly