Oh, the joy of Meteor e2e testing !

I'm setting up e2e testing on my Meteor app, ciboulette.net.

I had done it in the past but after large UI changes i just removed the tests as they were completely stale and generally painful to work with : 

  • they were brittle and/or slow
  • they were a pain to edit
  • they didn't run in CI

I'm trying again, this time using playwright.

To make the tests less brittle, i had to disable some of the "smarts" of the app to make the tests work at high speed  :

  • Instead of deferring heavy workloads, I run them synchronously before returning from a method.
  • My modal windows wait for the method result to come back from the server (no optimistic UI magic)
  • I had some logic to ignore double clicks that I had to disable in e2e mode
  • I completely mocked external services (google maps, google image, emails, sms, etc..)

To make editing easier, I had to fix a few things :

  • I start the e2e server with a different meteor local file, on a different port, with dedicated settings
  • I don't use the headful mode because it always start in the wrong place.
  • I take screenshots at every step, outputting the screenshot to one file that refreshes automatically when viewed with eog (eye of gnome) 
  • I use nodemon to restart the tests when i change them (but not when I change any other file) 
  • I start both my headless tests and eog with npm-run-all

Running the tests in github ci

  • The CI scrip uses start-server-and-test to start the meteor server in e2e mode and run the headless tests against it
  • The e2e Ci script runs npm install, but all other scripts run npm install --production. Playwright is heavy and is not required to build the app so i've added it as a devDependency. The unit tests are lightweight so i've added jest as a normal dependency.

The remaining challenge I have is to create snapshots at different stages of the test (page url, db content, cookies) so that i can skip most of the e2e tests suite when editing a small part of it. It's tricky, involves hardcoded mongo urls, some mongorestore and mongodump run with execSync .. and not working yet. 

 

Extract of my package.json scripts :

"e2e:server": "METEOR_LOCAL_DIR=.meteor/e2e  meteor --port 4000 --settings settings.e2e.json",
"e2e:cli:showImages": "eog tests/e2e/last-test-state.png",
"e2e:cli:nodemon": "SNAPALL=true nodemon --watch tests/e2e/01_signup.js tests/e2e/01_signup.js",
"e2e:cli": "run-p e2e:cli:* ",
"e2e:ci:node": "node tests/e2e/01_signup.js ",
"e2e:ci": "start-server-and-test e2e:server http://localhost:4000 e2e:ci:node"