Add retry with exponential backoff for transient failures in external service calls

[RUKAYAT-CODER]

Overview

External service calls (email provider, payment gateway, CDN invalidation) fail immediately on transient errors (network blip, 503). There is no retry logic, so a 1-second network hiccup causes a user-visible payment or email failure that requires manual intervention.

Specifications

Features:

• Retry transient failures (5xx, network errors) with exponential backoff and jitter.
• Stop retrying for client errors (4xx) as they are not transient.

Tasks:

• Create a RetryPolicy utility using cockatiel or a custom implementation with max 3 retries, 1s base delay, 2x multiplier, 30s max delay, and full jitter.
• Apply RetryPolicy to EmailService, PaymentProviderService, and CdnService.
• Add a Prometheus counter external_call_retry_total{service, attempt}.
• Add unit tests for retry behavior with mocked transient failures.

Impacted Files:

• New src/common/utils/retry-policy.ts
• src/notifications/email/, src/payments/providers/, src/cdn/

Acceptance Criteria

• A single transient 503 from the email provider is retried transparently.
• After 3 consecutive failures, the error is propagated to the caller.
• Prometheus counter shows retry counts per service.

[cristianFleita]

@cristianFleita has applied to work on this issue as part of the Stellar Wave Program's 6th wave.

Hello! I'd like to do this one

My plan is to build a centralized RetryPolicy utility to handle the heavy lifting and I will wrap the EmailService, PaymentProviderService, and CdnService with this policy

ℹ️ Repo Maintainers: To accept this application, review their application or assign @cristianFleita to this issue.

[aliyudotdev]

@aliyudotdev has applied to work on this issue as part of the Stellar Wave Program's 6th wave.

Hi can I please take up on this issue ?

ℹ️ Repo Maintainers: To accept this application, review their application or assign @aliyudotdev to this issue.

[drips-wave]

Congratulations, @aliyudotdev! 🎉 Your application was accepted by the repo's maintainers, and the issue is due on June 30, 2026.

🧑‍💻 @aliyudotdev: Please resolve the issue such that the repo's maintainers have enough time to review your contribution before the due date. You'll earn Points for completing the issue on-time, which will make you eligible for a share of the Stellar Wave Program's reward pool.

Warning

When opening a PR, please link it to this issue to ensure it gets tracked accurately. Points are awarded when this issue is marked as completed by the maintainer.

🤠 Repo maintainers: Please keep an eye on the contributor's progress and review their work before the due date. You can manage this issue, including adjusting its complexity and points, here.

🌊 Happy Wave 🌊