Javascript E2E / UI Testing (or how to break up with Selenium and move on)
You know that something is wrong when your tests are slow, unstable and ignored. Did I mention unmaintainable and hard to write? And finally how easy/hard is to setup your testing environment?
Writing tests is usually painful and boring task, but I knew we can do it better. So lets check how we can do it in JS. I started evaluating few tools in order to show power of JS. Sure, JS ecosystem is so volatile, when you finish with reading of this post, maybe new/shiny tool will popup (or not).
Puppeteer
With Puppeteer you can automate literally everything. We can create UI tests, run web scraper, take screenshots, create pdfs, check code coverage and visually compare site. Puppeteer control Chromium browser directly via DevTools protocol ⚡.
One thing that bothers me is when writing UI tests we need to write a lot of code which makes tests verbose (waitForSelector, waitForNavigation, wait and wait some more). Don’t get me wrong, we are in full control, but we have to write a lot of noisy code which may lead to unmaintainable code.
Cypress
Cypress was built with testing in mind and it have great features. Cypress runs code in browser by controlling browser via it’s API. Website is loaded inside iframe and Cypress have access to real DOM elements. Cypress has nice element inspection, assertion pattern, side pan with executed test commands and DOM snapshotting. I couldn’t configure tests to run in parallel, so I just move on. Never the less, Cypress is great testing tool.
Then I fell in love with TestCafe
All other tools felt a little hacky and awkward, but when I started using TestCafe I felt like I was having superpowers when writing UI tests. TestCafe runs in Node. It uses proxy to inject scripts that emulates user actions and it have full control from there.
Talk is cheap, so let me show you the code.
I will use TodoMVC app and you can checkout test file on github repo.
npm i testcafe
First I created class with page selectors and fixture with initial page:
With async await tests looks so clean and sweet:
Add npm script and run it or type inside terminal:
"test": "./node_modules/.bin/testcafe chrome tests/"
Let’s assert deletion of a todo:
There are lot of nice features, we can assert how many children element has or assert css class of element. Personally, I don’t like that nth(0) is zero based, because it defers from css :nth-child(1), but I love this functional api.
With their nice action api, we can do a lot of stuff, like a real user:
Now to test completing all todos:
To make things more DRY (Don’t Repeat Yourself) we can use for-of loop with await, but we must use Node 9+:
Where tests are failing
If assertion fails we get a really nice error message, with line number and surrounding context. Here I made mistake in test, just to see this nice output:
Parallel as hell
TestCafe CLI options
Cross-browser support:
$ testcafe chrome tests/
$ testcafe firefox tests/
$ testcafe ie tests/
$ testcafe safari tests/
$ testcafe all tests/
Device emulation:
$ testcafe "chrome:emulation:device=iphone 6" tests/
Headless:
$ testcafe chrome:headless tests/
Screenshots when test fails:
$ testcafe chrome tests/ --screenshots-on-fails -s screenshots
Decrease test speed:
$ testcafe chrome tests/ --speed 0.5
Debug on test fail:
$ testcafe chrome tests/ --debug-on-fail
And the best of all - concurrency:
$ testcafe chrome tests/ --concurrency 8
Debugging
Since it is running from Node, server side version to debug testcafe is to run it with --inspect
or with --inspect-brk
and place debugger
operator anywhere in test code.
Client side version of debugging is put t.debug()
inside test and testcafe will pause execution. Another client side version is to run testcafe with--debug-mode
flag and we can continue test execution manually.
Running on CI
TestCafe has great documentation for popular CI. Travis is always my first choice when I need quick and great setup. You can check job log for my latest test execution. And here is .travis.yml:
Conclusion
Follow testing pyramid, don’t write too many UI tests, they are supposed to verify some of user flows. Write enough to be confident in your application.
Hope you like it, TestCafe is really awesome tool and it makes writing tests so beautiful.
Get cup of coffee, check TestCafe documentation and write those tests ☕