Javascript E2E / UI Testing (or how to break up with Selenium and move on)

Filip Vitas
ITNEXT
Published in
5 min readJun 10, 2018

--

Web browser automation tools

You know that something is wrong when your tests are slow, unstable and ignored. Did I mention unmaintainable and hard to write? And finally how easy/hard is to setup your testing environment?

Writing tests is usually painful and boring task, but I knew we can do it better. So lets check how we can do it in JS. I started evaluating few tools in order to show power of JS. Sure, JS ecosystem is so volatile, when you finish with reading of this post, maybe new/shiny tool will popup (or not).

Puppeteer

With Puppeteer you can automate literally everything. We can create UI tests, run web scraper, take screenshots, create pdfs, check code coverage and visually compare site. Puppeteer control Chromium browser directly via DevTools protocol ⚡.
One thing that bothers me is when writing UI tests we need to write a lot of code which makes tests verbose (waitForSelector, waitForNavigation, wait and wait some more). Don’t get me wrong, we are in full control, but we have to write a lot of noisy code which may lead to unmaintainable code.

Cypress

Cypress was built with testing in mind and it have great features. Cypress runs code in browser by controlling browser via it’s API. Website is loaded inside iframe and Cypress have access to real DOM elements. Cypress has nice element inspection, assertion pattern, side pan with executed test commands and DOM snapshotting. I couldn’t configure tests to run in parallel, so I just move on. Never the less, Cypress is great testing tool.

Then I fell in love with TestCafe

All other tools felt a little hacky and awkward, but when I started using TestCafe I felt like I was having superpowers when writing UI tests. TestCafe runs in Node. It uses proxy to inject scripts that emulates user actions and it have full control from there.

Talk is cheap, so let me show you the code.

I will use TodoMVC app and you can checkout test file on github repo.

npm i testcafe

First I created class with page selectors and fixture with initial page:

With async await tests looks so clean and sweet:

Add npm script and run it or type inside terminal:

"test": "./node_modules/.bin/testcafe chrome tests/"

Let’s assert deletion of a todo:

There are lot of nice features, we can assert how many children element has or assert css class of element. Personally, I don’t like that nth(0) is zero based, because it defers from css :nth-child(1), but I love this functional api.

With their nice action api, we can do a lot of stuff, like a real user:

Now to test completing all todos:

To make things more DRY (Don’t Repeat Yourself) we can use for-of loop with await, but we must use Node 9+:

Where tests are failing

If assertion fails we get a really nice error message, with line number and surrounding context. Here I made mistake in test, just to see this nice output:

Parallel as hell

Running tests in parallel
Serial execution(left) vs Parallel execution (right)

TestCafe CLI options

Cross-browser support:

$ testcafe chrome tests/
$ testcafe firefox tests/
$ testcafe ie tests/
$ testcafe safari tests/
$ testcafe all tests/

Device emulation:

$ testcafe "chrome:emulation:device=iphone 6" tests/

Headless:

$ testcafe chrome:headless tests/

Screenshots when test fails:

$ testcafe chrome tests/ --screenshots-on-fails -s screenshots

Decrease test speed:

$ testcafe chrome tests/ --speed 0.5

Debug on test fail:

$ testcafe chrome tests/ --debug-on-fail

And the best of all - concurrency:

$ testcafe chrome tests/ --concurrency 8

Debugging

Since it is running from Node, server side version to debug testcafe is to run it with --inspect or with --inspect-brk and place debugger operator anywhere in test code.
Client side version of debugging is put t.debug() inside test and testcafe will pause execution. Another client side version is to run testcafe with
--debug-mode flag and we can continue test execution manually.

Running on CI

TestCafe has great documentation for popular CI. Travis is always my first choice when I need quick and great setup. You can check job log for my latest test execution. And here is .travis.yml:

Conclusion

Follow testing pyramid, don’t write too many UI tests, they are supposed to verify some of user flows. Write enough to be confident in your application.

Hope you like it, TestCafe is really awesome tool and it makes writing tests so beautiful.

Get cup of coffee, check TestCafe documentation and write those tests ☕

--

--