Skip to main content

सतत

Keep reinventing

Selenium based frontend tests using Python and Docker

Goal of the post is:

Using python write frontend tests for your Website that can run on a remote server, to make sure the site is operational.

lets elaborate more on some terms: Frontend tests, remote server and write tests in python.

  1. Frontend tests or more like browser based tests. They can be used to instruct a browser to:

    1. Open a given URL.
    2. Make sure that webpage has certain key elements, like signup, login etc.
    3. Complete a new signup process.
    4. Login using newly created account.
    5. Confirm that once logged in, again certain attributes are specific to the newly created user.

    These steps should be automated. A popular tool of choice for doing this is Selenium. The project provides driver binaries for different browsers and API bindings with all popular programming languages. Given that our requirement is to write tests in python, we will use python bindings.

  2. Getting tests to run on Remote server. Or more like, setting up the tests in such a manner that they can be run on any environment. Lets breakdown what we mean by this:
    1. Download binary drivers for all the browsers you would test with.
    2. Get python bindings, dependencies installed.
    3. And run the tests in headless browsers. Generally when you run selenium based tests, you would notice a browser open up and do all the steps you have in your tests. That means a working display environment in which browsers can open and render the site. On remote servers, we don't have traditional user interface running. So we would run UI-based browser tests on a browser without its graphical interface. This is known as headless browser.

A headless browser is a great tool for automated testing and server environments where you don't need a visible UI shell.

Okay, how do we do this?

I will use docker and docker-compose for this, because:

  1. The solution can run on any user environment(operating systems) that supports docker based development.
  2. docker-compose can take care of our requirements of providing multiple headless browsers.

From official document:

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

Selenium project provides docker images for different browsers. It has a concept of Grid that:

is a smart proxy server that allows Selenium tests to route commands to remote web browser instances. Its aim is to provide an easy way to run tests in parallel on multiple machines.

In such a grid, we can create a selenium-hub that can route tests to different nodes running different browsers that are regestired with hub. We will see how to use docker-compose to setup such a grid, register nodes running specific browser and finally write tests that uses this grid to run frontend tests.

This is the directory structure: 

$ ls
docker-compose.yml  Dockerfile  entrypoint.sh  requirements.txt  webtests.py

Selenium's docker documentation talks about an example compose configuration: 

version: "3"
services:
  tests:
    build:
      context: .
      dockerfile: Dockerfile
    depends_on:
      - firefox
  selenium-hub:
    image: selenium/hub:3.141.59-20200409
    container_name: selenium-hub
    ports:
      - "4444:4444"
  firefox:
    image: selenium/node-firefox:3.141.59-20200409
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444

We have three services in this configuration, first one is tests, this would be our test setup, we will look at that in a moment. Second one is selenium-hub that creates a Hub that can "expose" access to different kinds of browsers. And lastly, third service is firefox service, that registers to to selenium-hub. It will be responsible for running tests in Firefox browser.

Given that this is docker land, we will write a Dockerfile that would create our tests service:

Dockerfile: 

FROM python:3.7

WORKDIR /opt

ADD webtests.py /opt
ADD entrypoint.sh /opt
ADD requirements.txt /opt

RUN python -m pip install --upgrade pip
RUN pip3 install -r requirements.txt

ENTRYPOINT ["/bin/bash", "-c", "/opt/entrypoint.sh"]

Our image is based on official python3.7 Docker image. We will add three additional files to that image. requirements.txt is needed to install packages to vanilla python3.7 to run our tests. requirements.txt 

selenium==3.141.0

entrypoint.sh is a bash file that would run the tests: 

#!/bin/bash
python -m unittest -v webtests.py

We will use example test from Selenium documentation. In our setUp we will add a BusyWait logic that waits for selenium-hub to become available. firefox driver is running inside a separate service, so we will use concept of remote WebDriver and connect to it via driver http://selenium-hub:4444/wd/hub: 

import time
import unittest

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from urllib3.exceptions import MaxRetryError

class TestPythonOrgSearch(unittest.TestCase):
    def setUp(self):
	while True:
	    try:
		self.driver = webdriver.Remote(
		    command_executor='http://selenium-hub:4444/wd/hub',
		    desired_capabilities=DesiredCapabilities.FIREFOX
		)
	    except (WebDriverException, MaxRetryError):
		print('Waiting for selenium hub to become available...')
		time.sleep(0.2)
	    else:
		print('Connected to the selenium hub.....')
		break

    def test_search_in_python_org(self):
	driver = self.driver
	driver.get("http://www.python.org")
	self.assertIn("Python", driver.title)
	elem = driver.find_element_by_name("q")
	elem.send_keys("pycon")
	elem.send_keys(Keys.RETURN)
	assert "No results found." not in driver.page_source

    def tearDown(self):
	self.driver.close()

if __name__ == "__main__":
    unittest.main()

That's it. To run this test do: 

docker-compose build; docker-compose run --rm tests; docker-compose down

How to run same tests with different browsers?

Let us add Chrome service to our selenium-hub:

docker-compose.yml: 

version: "3"
services:
  tests:
    build:
      context: .
      dockerfile: Dockerfile
    depends_on:
      - firefox
      - chrome
  selenium-hub:
    image: selenium/hub:3.141.59-20200409
    container_name: selenium-hub
    ports:
      - "4444:4444"
  chrome:
    image: selenium/node-chrome:3.141.59-20200409
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
  firefox:
    image: selenium/node-firefox:3.141.59-20200409
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444

Now we have to configure our tests so that they can run with both the browsers. Based on a stackoverflow conversation, I used environment variable to do that. Change the entrypoint.sh bash script to: 

#!/bin/bash

echo 'Running tests with firefox'
BROWSER=firefox python -m unittest -v webtests.py
echo 'Running tests with chrome'
BROWSER=chrome python -m unittest -v webtests.py

With this change, BROWSER is set as a environment variable and we can access it in our tests and switch the browsers: 

#/usr/bin/env python3
import os
import time
import unittest
import warnings

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from urllib3.exceptions import MaxRetryError

class TestPythonOrgSearch(unittest.TestCase):
    def setUp(self):
	warnings.simplefilter("ignore", ResourceWarning)
	if os.environ.get('BROWSER') == 'chrome':
	    browser = DesiredCapabilities.CHROME
	else:
	    browser = DesiredCapabilities.FIREFOX
	while True:
	    try:
		self.driver = webdriver.Remote(
		    command_executor='http://selenium-hub:4444/wd/hub',
		    desired_capabilities=browser
		)
	    except (WebDriverException, MaxRetryError):
		print('Waiting for selenium hub to become available...')
		time.sleep(0.2)
	    else:
		print('Connected to the selenium hub.....')
		break
    def test_search_in_python_org(self):
	driver = self.driver
	driver.get("http://www.python.org")
	self.assertIn("Python", driver.title)
	elem = driver.find_element_by_name("q")
	elem.send_keys("pycon")
	elem.send_keys(Keys.RETURN)
	assert "No results found." not in driver.page_source

    def tearDown(self):
	self.driver.close()

if __name__ == "__main__":
    unittest.main()

And as you run the tests again with same command: 

$ docker-compose build; docker-compose run --rm tests; docker-compose down
[...]
Running tests with firefox
test_search_in_python_org (webtests.PythonOrgSearch) ... Waiting for selenium hub to become available...
Waiting for selenium hub to become available...
Waiting for selenium hub to become available...
Connected to the selenium hub.....
ok

----------------------------------------------------------------------
Ran 1 test in 14.396s

OK
Running tests with chrome
test_search_in_python_org (webtests.PythonOrgSearch) ... Connected to the selenium hub.....
ok

----------------------------------------------------------------------
Ran 1 test in 4.637s

OK