chore: fix test caching (#1101) #6473

Job	Run time
Linter	2m 21s
Unit tests (macos-latest)	1m 9s
Unit tests (windows-latest)	1m 42s
Unit tests (ubuntu-latest)	46s
Format check	44s
Integration tests (ubuntu-latest)	47s
Integration tests (windows-latest)	1m 39s
Integration tests (macos-latest)	1m 6s
Build	1m 25s
E2E tests (macos-latest)	10m 13s
E2E tests (ubuntu-latest)	7m 9s
E2E tests (windows-latest)	12m 39s
	41m 40s

Provide feedback