Task Summary
Exponential-backoff retry is hand-rolled once per module, so there is no single util a reviewer can point a contributor at.
| Site |
Module |
Kind |
Knobs |
Utils.retry |
amber |
Future, waits on a Timer |
attempts, base backoff ms, timer, onRetry |
LakeFSStorageClient.retryWithBackoff |
common/workflow-core |
blocking |
maxAttempts, initialDelayMillis, injectable sleep |
FileService.awaitDependency |
file-service |
blocking |
maxAttempts, initialDelayMillis, injectable sleep, description |
All three do the same thing: run the operation, on failure wait, double the wait, give up after N attempts, surface the last failure as the cause.
They cannot share code today because of the module graph and one dependency:
amber -> common/workflow-core (amber sees workflow-core, not the reverse)
file-service -> common/workflow-core
com.twitter:util-core declared only in amber/build.sbt
So Utils (in amber) is invisible to the two blocking copies, and the Future/Timer variant cannot move down without dragging util-core with it.
Proposal:
- Host the blocking variant in
common/workflow-core -- it needs nothing but Thread.sleep, so no new dependency -- with the injectable-sleep seam both existing copies already use for tests.
- Migrate
LakeFSStorageClient.retryWithBackoff and FileService.awaitDependency onto it, keeping their current messages and interrupt handling (both restore the interrupt status and fail fast; that behavior must survive).
- Leave the
Future/Timer variant as Utils.retry in amber, cross-referenced from the blocking one, unless util-core also moves down later.
Related: #7088 introduced the async Utils.retry and pointed region termination at it; this issue is the part that PR deliberately left out to stay a fix.
Task Type
Task Summary
Exponential-backoff retry is hand-rolled once per module, so there is no single util a reviewer can point a contributor at.
Utils.retryamberFuture, waits on aTimerLakeFSStorageClient.retryWithBackoffcommon/workflow-coreFileService.awaitDependencyfile-serviceAll three do the same thing: run the operation, on failure wait, double the wait, give up after N attempts, surface the last failure as the cause.
They cannot share code today because of the module graph and one dependency:
So
Utils(inamber) is invisible to the two blocking copies, and theFuture/Timervariant cannot move down without draggingutil-corewith it.Proposal:
common/workflow-core-- it needs nothing butThread.sleep, so no new dependency -- with the injectable-sleep seam both existing copies already use for tests.LakeFSStorageClient.retryWithBackoffandFileService.awaitDependencyonto it, keeping their current messages and interrupt handling (both restore the interrupt status and fail fast; that behavior must survive).Future/Timervariant asUtils.retryinamber, cross-referenced from the blocking one, unlessutil-corealso moves down later.Related: #7088 introduced the async
Utils.retryand pointed region termination at it; this issue is the part that PR deliberately left out to stay a fix.Task Type