Skip to content

[DRAFT] Replace Playwright with Lightpanda for browser tests#2606

Closed
mroderick wants to merge 3 commits intocodebar:masterfrom
mroderick:lightpanda-test
Closed

[DRAFT] Replace Playwright with Lightpanda for browser tests#2606
mroderick wants to merge 3 commits intocodebar:masterfrom
mroderick:lightpanda-test

Conversation

@mroderick
Copy link
Copy Markdown
Collaborator

@mroderick mroderick commented May 4, 2026

Summary

This PR tests replacing Playwright with Lightpanda for browser-dependent tests in CI.

Changes

  • Add capybara-lightpanda gem for Capybara integration
  • Update CI workflow to download and cache Lightpanda binary
  • Remove experimental gems (cuprite, selenium-webdriver, lightpanda-ruby)
  • Add spec/support/lightpanda.rb for driver configuration

Usage

# Run tests with Lightpanda
LIGHTPANDA=true bundle exec rspec spec/features/

Benefits

  • Lighter weight: ~65MB binary vs ~150MB+ Chromium download
  • Simpler CI: No complex browser installation, just download a single binary
  • Self-contained: Built-in CDP client, no external dependencies
  • Performance: ~20% faster test execution when it works

Limitations Found ⚠️

TomSelect/JavaScript Compatibility: Lightpanda's JavaScript engine doesn't fully support complex DOM manipulation libraries like TomSelect. Tests using TomSelect dropdowns fail because the library doesn't initialize properly.

Browser TomSelect Works Test Pass Rate
Playwright ✅ Yes 100%
Lightpanda ❌ No ~33%

Root Cause:

  • Playwright: tsWrapperClass="ts-wrapper tom-select single..."
  • Lightpanda: tsWrapperClass="N/A"

This is a fundamental limitation of Lightpanda's JavaScript engine (QuickJS) compared to full Chrome V8.

Local Testing Results

Based on 6 runs of 12 browser-dependent tests:

Browser Average Median Std Dev
Playwright 14.33s 16.30s 3.82s
Lightpanda 15.17s 16.43s 2.96s

CI Results

Run Playwright Lightpanda Notes
Group 1 ~2m3s 1m30s ✅ Both pass
Group 5 ~4m5s 3m20s ✅ Both pass
Groups 0,2,3,4 Pass Fail ❌ TomSelect issues

Conclusion

Not ready for production - Lightpanda shows promise with ~20% faster execution on compatible tests, but the JavaScript compatibility issues prevent full test suite passage.

Recommendation:

  • Keep Playwright as primary CI driver
  • Lightpanda could be used for specific test subsets without complex JS
  • Revisit when Lightpanda's JS engine matures

Status: DRAFT - Experimental, do not merge

mroderick added 3 commits May 4, 2026 21:09
This commit adds support for using the Lightpanda headless browser
instead of Playwright for browser-dependent tests.

Changes:
- Add capybara-lightpanda gem to the test group
- Configure CI workflow to download and cache Lightpanda binary
- Add spec/support/lightpanda.rb for Capybara driver configuration
- Remove experimental gems (cuprite, selenium-webdriver, lightpanda-ruby)

Usage:
  LIGHTPANDA=true bundle exec rspec spec/features/

The LIGHTPANDA_PATH environment variable can be used to specify
the path to the lightpanda binary (defaults to auto-download).

Benefits:
- Lighter weight (~65MB binary vs ~150MB+ Chromium)
- No complex browser installation in CI
- Self-contained CDP client
- Comparable test performance to Playwright
- Use the 'from' parameter as element ID instead of hardcoded value
- Add longer wait times when LIGHTPANDA=true for JavaScript execution
- Fixes test failures in managing_meeting_invitations_spec
@mroderick
Copy link
Copy Markdown
Collaborator Author

This was a fun experiment, good learnings. One day Lightpanda will be good enough to run our tests

@mroderick mroderick closed this May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant