End-to-end from front-end to back-end with Catcher
Catcher’s external modules 5.1.0 were finally released. It’s great news as it enables Selenium step for Front-end testing!
How should proper e2e test look like?
Imagine you have a user service with nice UI, which allows you to get information about users, registered in your system. Deeply in the back-end you also have an audit log, which saves all actions.
Before 5.1.0 you could use HTTP calls to mimic front-end behavior to trigger some actions on the back-end side.
Your test probably looked like:
- call http endpoint to search for a user
- check search event was saved to the database
- compare found user with search event, saved in the database
This test checks 100% of back-end functionality. But most likely front-end is the part of your system also! So proper end-to-end test should start with front-end application and end up in a back-end.
Without touching front-end you could have false-positive results in e2e tests. F.e.: a user has some special symbols in his name. All back-end tests passes and you deploy your application in production. After the deploy your users start to complain that front-end part of the application crashes. The reason is — front-end can’t handle back-end’s response when rendering user details with special symbols in his name.
With the new Catcher’s version you can include Front-end in your test. So — instead of calling http you can use selenium step.
The test
Let’s write a test, which will search for a user and will check that our search attempt was logged.
Every test starts with variables. To cover false-positive results we need to save multiple users and then check that only the correct one is returned. Let’s compose our users. Every user will have a random email and random name thanks to random built-in function.
variables:
users:
- name: '{{ random("name") }}'
email: '{{ random("email") }}'
- name: '{{ random("name") }}'
email: '{{ random("email") }}'
- name: '{{ random("name") }}'
email: '{{ random("email") }}'
Now we are ready to write our steps
.
Populate the data
The first step we need to do is to populate the data with prepare step.
Let’s prepare a users.sql
which will create all back-end tables (in case of clean run we don't have them).
CREATE TABLE if not exists users_table(
email varchar(36) primary key,
name varchar(36) NOT NULL
);
Next — we need to fill our table with test data. users.csv
will use our users
variable to prepare data for our step.
email,name
{%- for user in users -%}
{{ user.email }},{{ user.name }}
{%- endfor -%}
The step itself will take users.sql
and create database tables if needed. Then it will populate it using users.csv
based on users
variable.
steps:
- prepare:
populate:
postgres:
conf: '{{ postgres }}'
schema: users_table.sql
data:
users: users.csv
name: Populate postgres with {{ users|length }} users
Select a user to search for
The next (small) step is to select a user for our search. Echo step will randomly select user from users
variable and register it's email as a new variable.
- echo:
from: '{{ random_choice(users).email }}'
register: {search_for: '{{ OUTPUT }}'}
name: 'Select {{ search_for }} for search'
Search front-end for our user
With the Selenium step we can use our front-end to search for the user. Selenium step runs the script in JS/Java/Jar/Python from resources directory.
It passes Catcher’s variables as environment variables to the script so you can access it within Selenium. It also greps the script’s output, so you can access everything in Catcher’s next steps.
- selenium:
test:
file: register_user.js
driver: '/usr/lib/geckodriver'
register: {title: '{{ OUTPUT.title }}'}
The script will run register_user which searches for our selected user and will register page’s title.
Check the search log
After we did the search we need to check if it was logged. Imagine our back-end uses MongoDB. So we’ll use mongo step.
- mongo:
request:
conf: '{{ mongo }}'
collection: 'search_log'
find: {'text': '{{ search_for }}'}
register: {search_log: '{{ OUTPUT }}'}
This step searches MongoDB search_log
collection for any search attempts with our user in text.
Compare results
Final steps are connected with results comparison. First — we’ll use echo
again to transform our users
so that we can search in users by email.
- echo:
from: '{{ users|groupby("email")|asdict }}'
register: {users_kv: '{{ OUTPUT }}'}
Second — we will compare front-end page title got from selenium with MongoDB search log and user’s name.
- check:
and:
- equals: {the: '{{ users_kv[search_for][0].name }}', is: '{{ title }}'}
- equals: {the: '{{ title }}', is: '{{ search_log.name }}'}
The selenium resource
Let’s add a Selenium test resource. It will go to your site and will searches for your user. If everything is OK page title will be the result of this step.
Javascript
Selenium step supports Java, JS, Python and Jar archives. In this article I’ll show you all of them (except Jar, it is the same as Java, but without compilation). Let’s start with JavaScript.
const {Builder, By, Key, until} = require('selenium-webdriver');
async function basicExample() {
let driver = await new Builder().forBrowser('firefox').build();
try{
await driver.get(process.env.site_url);
await driver.findElement(By.name('q')).sendKeys(process.env.search_for, Key.RETURN);
await driver.wait(until.titleContains(process.env.search_for), 1000);
await driver.getTitle().then(function(title) {
console.log('{\"title\":\"' + title + '\"}')
});
driver.quit();
}
catch(err) {
console.error(err);
process.exitCode = 1;
driver.quit();
}
}
basicExample();
Catcher passes all it’s variables as environment variables, so you can access them from JS/Java/Python. process.env.site_url
in this example takes site_url from Catcher's variables and process.env.search_for
takes user email to search for it.
Everything you write to STDOUT is caught by Catcher. In case of JSON it will be returned as dictionary. F.e. with console.log('{\"title\":\"' + title + '\"}')
statement OUTPUT.title
will be available on Catcher's side. If Catcher can't parse JSON - it will return a text as OUTPUT
.
Python
Here is the Python implementation of the same resource. It should be also placed in resources
directory. To use it instead of Java implementation you need to change file
parameter in Selenium step.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import os
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = True
driver = webdriver.Firefox(options=options)
try:
driver.get(os.environ['site_url'])
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.clear()
elem.send_keys(os.environ['search_for'])
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
print(f'{"title":"{driver.title}"')
finally:
driver.close()
Java
Java is a bit more complex, as (if you are not using already compiled Jar) Catcher should compile Java source before running it. For this you need to have Java and Selenium libraries installed in your system.
Luckily Catcher comes with Docker image where libraries (JS, Java, Python), Selenium drivers (Firefox, Chrome, Opera) and tools (NodeJS, JDK, Python) installed.
package selenium;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxBinary;
import org.openqa.selenium.firefox.FirefoxOptions;
public class MySeleniumTest {
public static void main(String[] args) {
FirefoxBinary firefoxBinary = new FirefoxBinary();
FirefoxOptions options = new FirefoxOptions();
options.setBinary(firefoxBinary);
options.setHeadless(true);
WebDriver driver = new FirefoxDriver(options);
try {
driver.get(System.getenv("site_url"));
WebElement element = driver.findElement(By.name("q"));
element.sendKeys(System.getenv("search_for"));
element.submit();
System.out.println("{\"title\":\""+driver.getTitle() + "\"}");
} finally {
driver.quit();
}
}
}
Conclusion
Catcher’s update 5.1.0 unites front and back-end testing, allowing them both to exist in one testcase. It improves the coverage and make the test really end-to-end.