Make `AutoscaledPool` internal

[janbuchar]

• Follow-up to refactor!: Split off ConcurrencySystem from AutoscaledPool #3917 — with ConcurrencySystem owning all the concurrency/scaling configuration, AutoscaledPool is left as a bare task loop that BasicCrawler always constructs itself
• No reason to keep a generic parallel task runner in the public contract of a scraping library
• @internal on the class, AutoscaledPoolOptions and AutoscaledPoolPredicateOptions — same treatment Snapshotter/SystemStatus just got, the export stays for @crawlee/basic
• IConcurrencySystem and ConcurrencyConsumer stay public, custom governors are still a supported story
• Breaking, so 4.0 or never

Needs

• crawler-level pause()/resume()/abort() — autoscaledPool is public mostly because that's the only way to pause a crawler
• read-only concurrencySystem getter on the crawler — desiredConcurrency/currentConcurrency are otherwise unreachable when the crawler owns the default system
• rename autoscaledPoolOptions, or hoist isTaskReadyFunction/isFinishedFunction to top-level crawler options
• accept that maybeRunIntervalSecs/taskTimeoutSecs become unreachable — the crawler never forwarded them anyway
• check that apify-sdk-js doesn't build a pool of its own
• rewrite the AutoscaledPool part of the v4 upgrading guide, fix the stale ApiLink in guides/configuration.mdx

[janbuchar]

Counterpoint — maybe exposing a generic parallel executor is actually helpful in a web scraping library. @barjin @B4nan?

[barjin]

From my POV, this is encroaching on the bluebird / underscore etc. territory. I'm still on board with the original issue's reasoning.

Besides (and this is very much a guess), I believe the intersect of [Crawlee users] and [JS power users needing a parallel executor] is minimal.

[B4nan]

Yeah, to me it's an implementation detail. Power users can always bring their own pool, or even use the one we have. Making it internal doesn't mean it won't be exported, right?

Let's make the cut and reconsider if there is pushback from the users, I think we will identify this early in the v4 focus group anyway.

[janbuchar]

Yeah, to me it's an implementation detail. Power users can always bring their own pool, or even use the one we have. Making it internal doesn't mean it won't be exported, right?

The difference would be that we wouldn't need to worry that much about being backwards compatible — nobody forbids you to pull an internal symbol, but if it breaks for you, don't expect a shoulder you can cry on.