Oh, the joy of Meteor e2e testing !
I'm setting up e2e testing on my Meteor app, ciboulette.net.
I had done it in the past but after large UI changes i just removed the tests as they were completely stale and generally painful to work with :
- they were brittle and/or slow
- they were a pain to edit.
- they didn't run in CI
I'm trying again, this time using playwright.
To make the tests less brittle, i had to disable some of the "smarts" of the app to make the tests work at high speed :
- Instead of deferring heavy workloads, I run them synchronously before returning from a method.
- My modal windows wait for the method result to come back from the server (no optimistic UI magic)
- I had some logic to ignore double clicks that I had to disable in e2e mode
- I completely mocked external services (google maps, google image, emails, sms, etc..)
To make editing easier, I had to fix a few things :
- I start the e2e server with a different meteor local file, on a different port, with dedicated settings
- I don't use the headful mode because it always start in the wrong place.
- I take screenshots at every step, outputting the screenshot to one file that refreshes automatically when viewed with eog (eye of gnome)
- I use nodemon to restart the tests when i change them (but not when I change any other file)
- I start both my headless tests and eog with npm-run-all
Running the tests in github ci
- The CI scrip uses start-server-and-test to start the meteor server in e2e mode and run the headless tests against it
- The e2e Ci script runs npm install, but all other scripts run npm install --production. Playwright is heavy and is not required to build the app so i've added it as a devDependency. The unit tests are lightweight so i've added jest as a normal dependency.
The remaining challenge I have is to create snapshots at different stages of the test (page url, db content, cookies) so that i can skip most of the e2e tests suite when editing a small part of it. It's tricky, involves hardcoded mongo urls, some mongorestore and mongodump run with execSync .. and not working yet.
Extract of my package.json scripts :
"e2e:server": "METEOR_LOCAL_DIR=.meteor/e2e meteor --port 4000 --settings settings.e2e.json",
"e2e:cli:showImages": "eog tests/e2e/last-test-state.png",
"e2e:cli:nodemon": "SNAPALL=true nodemon --watch tests/e2e/01_signup.js tests/e2e/01_signup.js",
"e2e:cli": "run-p e2e:cli:* ",
"e2e:ci:node": "node tests/e2e/01_signup.js ",
"e2e:ci": "start-server-and-test e2e:server http://localhost:4000 e2e:ci:node"