feat: framework refactor + decouple from Hyperf#349
feat: framework refactor + decouple from Hyperf#349binaryfire wants to merge 3761 commits intohypervel:0.4from
Conversation
|
@albertcht To illustrate how much easier it will be to keep Hypervel in sync with Laravel after this refactor, I asked Claude how long it would take to merge laravel/framework#58461 (as an example) into this branch. This is what it said: So just 5-10 minutes of work with the help of AI tooling! Merging individual PRs is inefficient - merging releases would be better. I can set up a Discord channel where new releases are automatically posted via webhooks. Maybe someone in your team can be responsible for monitoring that channel's notifications and merging updates ever week or 2? I'll only be 1-2 hours of work once the codebases are 1:1. We should be diligent about staying on top of merging updates. Otherwise we'll end up in in the same as Hyperf - i.e. the codebase being completely out of date with the current Laravel API. |
|
Hi @binaryfire , Thank you for submitting this PR and for the detailed explanation of the refactor. After reading through it, I strongly agree that this is the best long-term direction for Hypervel. Refactoring Hypervel into a standalone framework and striving for 1:1 parity with Laravel will indeed solve the current issues regarding deep coupling with Hyperf, maintenance difficulties, outdated versions, and inefficient AI assistance. While this is a difficult step, it is absolutely necessary for the future of the project. Regarding this refactor and the planning for the v0.4 branch, I have a few thoughts to verify with you:
Thank you again for dedicating so much effort to driving this forward; this is a massive undertaking. Let's move forward gradually on this branch with ongoing Code Reviews. |
|
Hi @albertcht Thanks for the detailed response! I'm glad we're aligned on the direction. Let me address each point:
Let me know your thoughts! |
binaryfire
force-pushed
the
feature/hyperf-decouple
branch
from
8cec3bf to
bfffa6f
Compare
|
Hi @albertcht. The All the Laravel tests have been ported over and are passing (the unit tests, as well as the integration tests for MySQL, MariaDB, Postgres and SQLite). I've implemented Context-based coroutine safety, static caching for performance and modernised all the types. The code passes PHPStan level 5. Let me know if there's anything I've missed, if you have any ideas or you have any questions. The other packages aren't ready for review yet - many of them are mid-migration and contain temporary code. So please don't review the others yet :) I'll let you know when each one is ready. A few points:
|
binaryfire
force-pushed
the
feature/hyperf-decouple
branch
from
80f3ef2 to
94c115e
Compare
|
@albertcht The following packages are ready for review. I've modernised typing, optimised the code, added more tests (including integration tests) and fixed several bugs.
I've also ported https://github.com/friendsofhyperf/redis-subscriber into the Redis package. The subscription methods were all blocking - now they're coroutine friendly. With the previous implementation, if you wrapped
The approach follows the same pattern suggested in hyperf/hyperf#4775 (https://github.com/mix-php/redis-subscriber, which Deeka ported to https://github.com/friendsofhyperf/components). I.e. a dedicated raw socket connection with This is a good article re: this issue for reference: https://openswoole.com/article/redis-swoole-pubsub |
|
Hi @albertcht! The new This is Swoole-optimised version of Laravel's IoC Container, replacing Hyperf's container. The goal: give Hypervel the complete Laravel container API while maintaining performance parity with Hyperf's container and full coroutine safety for Swoole's long-running process model. Why replace Hyperf's container?Hyperf's container is minimal. It exposes
Also, the API is very different to Laravel's. This makes it difficult to port Laravel code or use Laravel's service provider patterns without shimming everything. The new container closes that gap completely and makes interacting with the container much more familiar to Laravel devs. It also means that our package and test code will be closer to 1:1 with Laravel now. APIThe new container implements the full Laravel container contract:
It also supports closure return-type bindings (register a binding by returning a typed value from a closure, including union types), Key API difference from HyperfLike Hyperf's Auto-singletoned instances are stored in a separate Attribute-based injection16 contextual attributes are included, providing declarative dependency injection:
Example: class OrderService
{
public function __construct(
#[Config('orders.tax_rate')] private float $taxRate,
#[Tag('payment-processors')] private array $processors,
#[Authenticated] private User $user,
) {}
}PerformanceBuild recipe cachingConstructor parameters are analyzed via reflection once per class and cached as Method parameter caching
Reflection caching
Hot-path optimizations
Performance vs HyperfThe singleton cache-hit path does marginally more work than Hyperf's single Coroutine safetyAll per-request state is stored in coroutine-local
Circular dependency detection uses two complementary mechanisms:
All transient Context state is cleaned up in Scoped instance cleanup is handled consistently across all invalidation paths. Tests~220 tests:
Everything passes at PHPStan level 5. Let me know what you think |
src/database/src/Listeners/RegisterConnectionResolverListener.php
Outdated
Show resolved
Hide resolved
getBaseRedisDb() defaulted to 1, but database.php defaults to 0. When REDIS_DB env var was unset, the trait and config disagreed — the trait created connections on DB 1 while rawRedisClientWithoutPrefix read from config's DB 0, causing key existence checks to fail. Change default from 1 to 0 to match database.php. Remove REDIS_DB from redis workflow since the defaults now agree without it.
Same change as copyFromNonCoroutine() — replace exchangeArray() with per-key assignment so existing values in the target coroutine are preserved. All current callers run on fresh coroutines where the result is identical, but merge is safer for any future use case where the target already has state.
…erability Hypervel workers can now process jobs dispatched by Laravel and vice versa when both apps share the same queue backend and cache prefix. This enables Laravel apps to offload job processing to Hypervel for Swoole concurrency without migrating the entire application. Changes: - Use Illuminate\Queue\CallQueuedHandler@call in job payloads - Align operational cache keys (pause, restart) with Laravel's illuminate: prefix - Align middleware lock prefixes (overlap, throttle) with Laravel's laravel prefix - Align unique job lock prefix and context keys with Laravel's laravel_unique_job prefix - Register class aliases for CallQueuedHandler and ModelIdentifier so Hypervel can resolve Illuminate class references from Laravel-dispatched payloads - Add LaravelInteropTest to guard all interop-critical values against regression
…ean up aliases - Rename copy() to copyFrom() in both Context and WebSocketServer\Context for clarity — it copies *from* another coroutine/FD into the current one - Fix WebSocketServer\Context::copyFrom() to merge instead of replacing the target FD's context, preserving existing values (bug fix) - Add test for WebSocket context merge-preservation behavior - Rename CoContext alias to CoroutineContext, WsContext to WebSocketContext - Update all callers, comments, and docblocks
…ontext Free up the Context name for Laravel's Context (propagated context for jobs, logs, etc.) which will be ported in a future session. - Rename Hypervel\Context\Context → CoroutineContext - Rename Hypervel\Context\ParentContext → ParentCoroutineContext - Delete context() helper (name reserved for Laravel's context()) - Update all imports, FQCN references, and call sites (143 files)
The Logger's withContext() feature stored per-channel context under `__log.context`, which is too generic and will collide with the new Context repository key. Rename to `__log.channel_context` to clearly describe what it stores. Also rename misleading local array keys in LogLoggerTest that used the `__log.` prefix but were unrelated to coroutine context.
Objects stored in coroutine context that need deep-copying when context is copied between coroutines can implement this interface. This replaces the hard-coded PropagatedContext check in CoroutineContext::copyFrom().
Remove propagated(), hasPropagated(), and PROPAGATED_CONTEXT_KEY from CoroutineContext. Replace hard-coded instanceof PropagatedContext checks in copyFrom() and copyFromNonCoroutine() with the generic ReplicableContext interface. CoroutineContext no longer has any knowledge of the Context feature.
Marker interface extending Monolog's ProcessorInterface, matching Laravel's Illuminate\Contracts\Log\ContextLogProcessor. Allows apps to swap the log processor implementation.
Move PropagatedContext to Hypervel\Log\Context\Repository, matching Laravel's Illuminate\Log\Context\Repository. Move events to Hypervel\Log\Context\Events. Move ContextLogProcessor into the Context subdirectory and implement the new contract. Add ContextServiceProvider with queue payload hooks (using illuminate:log:context key for cross-framework interop) and ContextLogProcessor contract binding. Remove unused dependencies from context package composer.json now that PropagatedContext no longer lives there.
Provides static access to the coroutine-local Context Repository, matching Laravel's Illuminate\Support\Facades\Context. Overrides resolveFacadeInstance() to bypass the container and resolve directly from Repository::getInstance(), avoiding unnecessary overhead. The $resolvedInstance check is preserved for mock/swap support in tests.
Add to Application::registerBaseServiceProviders() matching Laravel's pattern. Also add to composer.json extra.hypervel.providers for standalone package installs. Add hypervel/queue as explicit dependency of the log package for the ContextServiceProvider's queue hooks.
- QueueServiceProvider: remove configurePropagatedContext(), now handled by ContextServiceProvider - CallQueuedHandler: use Repository::hasInstance()/getInstance() - InteractsWithUniqueJobs: use Context facade, matching Laravel - LogManager: resolve ContextLogProcessor via contract
RoutingServiceProvider is a base provider registered in Application::registerBaseServiceProviders(). Laravel does not include base providers in DefaultProviders — they are loaded earlier in the bootstrap sequence and do not need to be discovered.
Move all PropagatedContext tests from tests/Context/ to tests/Log/, matching the package they now test. Rename test files and classes to reflect the new Repository/Context naming. Update all imports, access patterns, and payload key references. - PropagatedContextTest → ContextTest - PropagatedContextCoroutineTest → ContextCoroutineTest - PropagatedContextLogTest → ContextLogProcessorTest - PropagatedContextQueueTest → ContextQueueTest - ContextPropagatedTest → ContextInstanceTest - PropagatedContextIntegrationTest → ContextIntegrationTest (Integration/Log/) - LogManagerTest: update ContextLogProcessor import - CallQueuedHandlerTest: update context access - AfterEachTestSubscriber: update flushState reference
Update the Context container attribute to use Hypervel\Log\Context\Repository::getInstance() instead of resolving through the container. Remove the @todo comment. Port the full ContextualAttributeBindingTest from Laravel, covering all contextual attribute types (Auth, Cache, Config, Context, Database, Give, Log, RouteParameter, Storage, Tag). Context tests adapted to use Repository::getInstance() directly.
Uncomment and implement the ContextIntegration that adds log context data to Sentry events. Uses Repository::hasInstance() to avoid allocating an empty Repository when no context exists. Port tests from sentry-laravel's LaravelContextIntegrationTest, covering: integration registration, exception capture with/without context, hidden context exclusion, and transaction capture.
Hyperf's annotation scanning (#[Process], #[Listener], etc.) is intentionally not being ported — Hypervel uses explicit registration via service providers and config. The TODO was waiting for a system that will never exist.
The listen() loop in AbstractProcess broke out on any SocketAcceptException, including transient errors like EINTR (signal interruption) and EAGAIN. This left the process running but permanently deaf to pipe messages. Now only permanent errors (peer closed pipe, EBADF, ECONNRESET, etc.) break the loop. Transient errors (EINTR, EAGAIN) are logged and retried on the next iteration. This prevents both silent message loss and hot-spinning on dead sockets. Adds SocketAcceptException::isPermanent() to distinguish the two cases, and extracts getListenSocket() for testability.
Major alignment of the Hypervel sentry package structure, config, and source code with the upstream sentry-laravel package while preserving all Swoole-specific adaptations. Directory structure: - Integrations/ → Integration/ (singular, matching upstream) - Integration.php moved to package root (matching upstream) - Traits/ → Features/Concerns/ (matching upstream) - Commands/ → Console/ (matching upstream) - RequestFetcher → Http/HypervelRequestFetcher (matching upstream pattern) Config: - Aligned all defaults with upstream (traces_sample_rate=null, etc.) - Removed enable_tracing (SDK handles via traces_sample_rate) - Fixed env var naming to match upstream _ENABLED suffix pattern - Changed DSN env var to SENTRY_HYPERVEL_DSN - Added missing upstream keys (org_id, strict_trace_continuation, etc.) - Separated Hypervel-specific config (pool, features, integrations) Source: - Deleted redundant RequestIntegration (duplicated SetRequestIpMiddleware) - Moved DB query breadcrumbs from DbQueryFeature into EventHandler - Added SQL transaction breadcrumbs to EventHandler (Hypervel addition) - Added Sanctum TokenAuthenticated support to EventHandler - Added email string cast and null message check from upstream - Added DebugFileLogger singleton registration - Aligned Tracing\EventHandler constructor with upstream (array $config) - Aligned visibility modifiers with upstream (private vs protected) Test infrastructure: - Aligned SentryTestCase with upstream TestCase helpers - Added TestCaseExceptionHandler for exception capture in tests - Fixed startTransaction() double-init with idempotent initSpanRecorder - Tests explicitly set traces_sample_rate when needed
External packages (e.g. the scheduler) need to reference this key to selectively propagate Context to child coroutines via Concurrent::fork().
Mirrors Concurrent::create() but uses Coroutine::fork() to copy specified parent context keys to the child coroutine. This enables selective context propagation (e.g. only the Context Repository) without over-copying request or Sentry state.
Use Concurrent::fork() with the Context Repository key so background scheduled tasks receive the parent's Context data (including hidden context like Sentry check-in IDs). This matches Laravel's __LARAVEL_CONTEXT env var propagation but via coroutine context copying instead of process environment variables.
- CacheFeature: add console session guard, null key handling, self:: constants - ConsoleSchedulingFeature: use Context hidden storage for check-in IDs, isApplicable() always true, remove cache dependency - QueueFeature: align with upstream structure, use payload(), remove dead constants and getJobName(), keep Hypervel-specific JobFailed handler - AboutCommand: convert to AboutCommandIntegration callable class, register via AboutCommand::add() matching upstream pattern - HypervelRequestFetcher: cache PSR-7 factories for worker lifetime - SentryServiceProvider: remove dead HttpClientInterface singleton, add AboutCommandIntegration registration
Register a Coroutine::defer() callback on first transport checkout to ensure checked-out transports are released back to the pool even if the coroutine terminates abnormally (timeout, uncaught exception) and FlushEventsMiddleware::terminate() never runs. The defer is registered once per coroutine on first send(). If close() already ran, the deferred callback sees an empty list and is a no-op.
Properties with strict type declarations and defaults can never be null, making ?? fallbacks dead code. Removed across foundation, http, testbench, and validation packages. Added nullCoalesce.property to HandleExceptions phpstan-ignore where Closure::call() rebinding makes the guard genuinely needed at runtime.
- Logs/LogChannel: use local Hypervel\Sentry\Logs\LogsHandler instead of Sentry\Monolog\LogsHandler (SDK handler). Matches upstream which uses its own package-local LogsHandler, not the SDK one. - RedisFeature: add console short-circuit to getSessionKey() to avoid unnecessary session/database boot in CLI contexts. Use 'session.store' binding instead of Session::class. Mirrors the same fix in CacheFeature.
Move finishTransaction() from middleware terminate() to a Coroutine::defer() registered in startTransaction(). Since defers run in LIFO order, registering early ensures the transaction finishes last — after all dispatchAfterResponse() work and other deferred callbacks have completed, capturing their spans. Previously, the transaction was finished during middleware termination, before Coroutine::defer() callbacks ran. Any spans created by after-response work (e.g. dispatchAfterResponse) were silently lost.
2 participants
Hi @albertcht. This isn't ready yet but I'm opening it as a draft so we can begin discussions and code reviews. The goal of this PR is to refactor Hypervel to be a fully standalone framework that is as close to 1:1 parity with Laravel as possible.
Why one large PR
Sorry about the size of this PR. I tried spreading things across multiple branches but it made my work a lot more difficult. This is effectively a framework refactor - the database package is tightly coupled to many other packages (collections, pagination, pool) as well as several support classes, so all these things need to be updated together. Splitting it across branches would mean each branch needs multiple temporary workarounds + would have failing tests until merged together, making review and CI impractical.
A single large, reviewable PR is less risky than a stack of dependent branches that can't pass CI independently.
Reasons for the refactor
1. Outdated Hyperf packages
It's been difficult to migrate existing Laravel projects to Hypervel because Hyperf's database packages are quite outdated. There are almost 100 missing methods, missing traits, it doesn't support nested transactions, there are old Laravel bugs which haven't been fixed (eg. JSON indices aren't handled correctly), coroutine safety issues (eg. model
unguard(),withoutTouching()). Other packages like pagination, collections and support are outdated too.Stringablewas missing a bunch of methods and traits, for example. There are just too many to PR to Hyperf at this point.2. Faster framework development
We need to be able to move quickly and waiting for Hyperf maintainers to merge things adds a lot of friction to framework development. Decoupling means we don't need to work around things like PHP 8.4 compatibility while waiting for it to be added upstream. Hyperf's testing package uses PHPUnit 10 so we can't update to PHPUnit 13 (and Pest 4 in the skeleton) when it releases in a couple of weeks. v13 has the fix that allows
RunTestsInCoroutineto work with newer PHPUnit versions. There are lots of examples like this.3. Parity with Laravel
We need to avoid the same drift from Laravel that's happened with Hyperf since 2019. If we're not proactive with regularly merging Laravel updates every week we'll end up in the same situation. Having a 1:1 directory and code structure to Laravel whenever possible will make this much easier. Especially when using AI tools.
Most importantly, we need to make it easier for Laravel developers to use and contribute to the framework. That means following the same APIs and directory structures and only modifying code when there's a good reason to (coroutine safety, performance, type modernisation etc).
Right now the Hypervel codebase is confusing for both Laravel developers and AI tools:
hypervel/contractspackage, the Hyperf database code is split across 3 packages, the Hyperf pagination package ishyperf/paginatorand nothyperf/pagination)static::registerCallback('creating')vsstatic::creating())ConfigProviderand LaravelServiceProviderpatterns across different packages is confusing for anyone who doesn't know HyperfThis makes it difficult for Laravel developers to port over apps and to contribute to the framework.
4. AI
The above issues mean that AI needs a lot of guidance to understand the Hypervel codebase and generate Hypervel boilerplate. A few examples:
hypervel/contractsfor contracts) and then have to spend a lot of time grepping for things to find them.And so on... This greatly limits the effectiveness of building Hypervel apps with AI. Unfortunately MCP docs servers and CLAUDE.md rules don't solve all these problems - LLMs aren't great at following instructions well and the sheer volume of Laravel data they've trained on means they always default to Laravel-style code. The only solution is 1:1 parity. Small improvements such as adding native type hints are fine - models can solve that kind of thing quickly from exception messages.
What changed so far
New packages
illuminate/databaseportilluminate/collectionsportilluminate/paginationportilluminate/contracts)hyperf/pool)Macroableto a separate package for Laravel parityRemoved Hyperf dependencies so far
Database package
The big task was porting the database package, making it coroutine safe, implementing performance improvements like static caching and modernising the types.
whereLike,whereNot,groupLimit,rawValue,soleValue, JSON operations, etc.Collections package
Contracts package
Support package
hyperf/tappable,hyperf/stringable,hyperf/macroable,hyperf/codecdependenciesStr,Envand helper classes from LaravelHypervel\Contextwrappers (will be portinghyperf/contextsoon)Number::useCurrency()wasn't actually setting the currency)Coroutine safety
withoutEvents(),withoutBroadcasting(),withoutTouching()now use Context instead of static propertiesUnsetContextInTaskWorkerListenerto clear database context in task workersConnection::resetForPool()to prevent state leaks between coroutinesDatabaseTransactionsManagercoroutine-safeBenefits
Testing status so far
What's left (WIP)
The refactor process
Hyperf's Swoole packages like
pool,coroutine,contextandhttp-serverhaven't changed in many years so porting these is straightforward. A lot of the code can be simplified since we don't need SWOW support. And we can still support the ecosystem by contributing any improvements we make back to Hyperf in separate PRs.Eventually I'll refactor the bigger pieces like the container (contextual binding would be nice!) and the config system (completely drop
ConfigProviderand move entirely to service providers). But those will be future PRs. For now the main refactors are the database layer, collections and support classes + the simple Hyperf packages. I'll just port the container and config packages as-is for now.Let me know if you have any feedback, questions or suggestions. I'm happy to make any changes you want. I suggest we just work through this gradually, as an ongoing task over the next month or so. I'll continue working in this branch and ping you each time I add something new.
EDIT: New comments are getting lost in the commit history so linking them here:
Updated
hypervel/contextpackage ready for reviewSee: #349 (comment)
New
hypervel/http-server,hypervel/http&hypervel/routingpackages ready for reviewSee: #349 (comment)
New
hypervel/containerpackage ready for reviewSee: #349 (comment)
New
hypervel/context,hypervel/coordinator,hypervel/coroutine,hypervel/engine,hypervel/pool&hypervel/redispackages ready for reviewSee: #349 (comment)
New
hypervel/databasepackage ready for reviewSee: #349 (comment)