Skip to content

feat: add Lightpanda support via new LightpandaCrawler package#3481

Draft
candidosales wants to merge 6 commits intoapify:masterfrom
candidosales:support-lightpanda
Draft

feat: add Lightpanda support via new LightpandaCrawler package#3481
candidosales wants to merge 6 commits intoapify:masterfrom
candidosales:support-lightpanda

Conversation

@candidosales
Copy link
Copy Markdown

@candidosales candidosales commented Mar 13, 2026

  • Introduced @crawlee/lightpanda package for integrating Lightpanda with Crawlee.
  • Implemented LightpandaCrawler for fast, low-memory headless crawling using the Chrome DevTools Protocol (CDP).
  • Created LightpandaLauncher and LightpandaPlugin to manage Lightpanda's lifecycle and connection.
  • Added support for automatic process management and proxy configuration.
  • Included documentation and changelog for the new package.
  • Added documentation on how to test it
  • Inspiration: https://github.com/vercel-labs/agent-browser/releases/tag/v0.17.0

Tests

# Launch the Docker
docker run -d --name lightpanda -p 9222:9222 lightpanda/browser:nightly

# Run the script - Path: packages/lightpanda-crawler/examples
cd packages/lightpanda-crawler/examples
LIGHTPANDA_AUTO_START=false tsx --tsconfig tsconfig.json scrape-books.ts
CleanShot 2026-03-12 at 22 53 06@2x CleanShot 2026-03-12 at 22 53 16@2x

- Introduced `@crawlee/lightpanda` package for integrating Lightpanda with Crawlee.
- Implemented `LightpandaCrawler` for fast, low-memory headless crawling using the Chrome DevTools Protocol (CDP).
- Created `LightpandaLauncher` and `LightpandaPlugin` to manage Lightpanda's lifecycle and connection.
- Added support for automatic process management and proxy configuration.
- Included comprehensive documentation and changelog for the new package.
@candidosales

This comment was marked as spam.

Copy link
Copy Markdown
Member

@barjin barjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, and thank you for your contribution, @candidosales!

After discussing this with the rest of the team, we are not planning to merge this PR right now. For the upcoming major release (v4), we are preparing changes to the Playwright/Puppeteer integrations that should allow us to connect to remote browser instances via CDP and the Playwright internal protocol (see here)

Once these changes are merged, integrating Lightpanda with Crawlee should be much easier and should not require most of the changes proposed in this PR.

Thank you for your understanding.

@candidosales
Copy link
Copy Markdown
Author

candidosales commented Mar 16, 2026

@barjin, great, thanks for the explanation! I'll wait for the next version :)

@candidosales candidosales marked this pull request as draft March 17, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants