Commit af1855f
fix(pii): install CUDA torch on amd64 so GLiNER can run on GPU
The published pii image installed a CPU-only torch build, so GLiNER on the
ECS GPU fleet died at model load with "Attempting to deserialize object on a
CUDA device but torch.cuda.is_available() is False". The Dockerfile already
had a TORCH_INDEX_URL arg, but no CI job ever passed --build-arg, so every
image silently took the cpu default.
Select the wheel index from TARGETARCH instead: amd64 gets cu128, arm64 keeps
the cpu index (cu128 publishes no aarch64 wheel at 2.11.0, and no arm64 target
has a GPU). CUDA torch falls back to CPU when no GPU is present, so one image
still serves both the Fargate CPU tasks and the EC2 GPU tasks off the same tag
— no CI or CDK changes needed.
cu128 keeps sm_75, the compute capability of the fleet's T4s, and its CUDA 12.8
runtime needs driver >=525 via minor-version compatibility, which the ECS GPU
AMI satisfies. cu121 was not an option: that index stops at torch 2.5.1.
Verified in an amd64 build of the changed block:
2.11.0+cu128 cuda=12.8 arch=sm_75 sm_80 sm_86 sm_90 sm_100 sm_120
arm64 still resolves to 2.11.0+cpu. A build-time assert now fails the image
if amd64 ever silently regresses to a cpu wheel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QHNEWVrh7k89m8Wtqzhs181 parent 60bea92 commit af1855f
1 file changed
Lines changed: 28 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
50 | 58 | | |
51 | | - | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
52 | 62 | | |
53 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
54 | 76 | | |
55 | 77 | | |
56 | 78 | | |
| |||
0 commit comments