Setting up testing with Selenium
Background
This is not an MDriven specific instruction - it's a page with short instructions and links to other pages to make it easy to know one way to follow to set up testing towards your system.
Selenium does testing of web sites, so it's targeting a Turnkey application. These instructions and testing have all been done on Windows 11.
Selenium is a very widely used framework for running scripted web tests. You can use it for both application testing and load testing. Load testing, though, will be for a small number of clients, because it takes a lot of CPU and memory on the test server to run parallel web browsers.
The easiest way to get going is using the Selenium IDE plug-in for your browser, for example, Chrome. With the IDE, you can create, edit and run a script without any setup on your local development machine. https://chromewebstore.google.com
Search in the Chrome web store for "Selenium IDE". Add it, and then click on the browser extension to start it.
Save your test projects (scripts) to a local folder of your choice.
Running your tests separate from your Desktop browser
Run tests on the Grid from the Selenium command line runner - read more here.
Install the Selenium command line runner Command Line Runner
- The command line runner uses node.js.
- Install the node.js from https://nodejs.org/en/download/ using the windows installer.
- Install the node.js from https://nodejs.org/en/download/ using the windows installer.
- Once node.js is installed, run npm install -g selenium-side-runner
Selenium Grid execution environment
When you have a script that does what you want it to do, as simple or complex as that needs to be, install the Selenium Grid execution environment to run many copies of your test, either one after another, or probably more importantly, in parallel to ensure good overall performance.
Installing a Selenium Grid is surprisingly simple, and we recommend this configuration (there are any number of ways, this is just a simple way on Windows):
- Install WSL2 on Windows (add feature to Windows, Windows Subsystem for Linux)
- Make sure your machine has vCPU turned on in BIOS.
- Install Docker Desktop
- Use the docker-selenium images/containers provided by SeleniumHQ on GitBub to run a whole grid setup without effort
- https://github.com/SeleniumHQ/docker-selenium (This page is very detailed and well written, please read it carefully, it gives most answers on which setup to run.)
Running your test script
In the folder you saved your tests (they are files with the extension .side), create a file called .side.yml, which will be used for all .side files, regardless of name. This single file is used by the command line runner (if you don't override that from the command line).
If your .side.yml file doesn't have a server: instruction, it will run with the locally installed browser, just like from the Selenium IDE. With a server: instruction, the side runner will call the grid server for execution.
# this is how your .side.yml should or could look like capabilities: browserName: "chrome" se:recordVideo: "true" se:screenResolution: "1920x1080" server: "http://localhost:4444/wd/hub"
Note that, for example, the browser name needs to be without capital letters, it's case sensitive, and nothing will work if you have "Chrome" there instead of "chrome".
If you want a video recording of all your test runs, add se:recordVideo: "true" to the .side.yml file like above.
Which Grid setup should you use?
There are grid setups that have only one node, or one node for each of the big browsers, Chrome, Firefox and Edge, but there is also a dynamic grid setup which we find very easy to use. It will automatically scale up to the number of CPU cores that your machine has.
To set up the entire dynamic grid, create a "config.toml" file with configuration next to your test script files ".side" and the ".side.yml" file, that configures browser selection and where to find the grid.
Shortened example:
[docker]
# Configs have a mapping between the Docker image to use and the capabilities that need to be matched to
# start a container with the given image.
configs = [
"selenium/standalone-firefox:4.16.1-20231219", '{"browserName": "firefox"}',
"selenium/standalone-chrome:4.16.1-20231219", '{"browserName": "chrome"}',
"selenium/standalone-edge:4.16.1-20231219", '{"browserName": "MicrosoftEdge"}'
]
# Windows: make sure Docker Desktop exposes the daemon via tcp, and use http://host.docker.internal:2375.
url = "http://127.0.0.1:2375"
# Docker image used for video recording
video-image = "selenium/video:ffmpeg-6.1-20231219"
To start this configuration from PowerShell, use this command. For ease of use, it's best to create a .ps1 file, for example, called StartDynamicGrid.ps1, with this content (it's multi-line and is tricky to keep at the command line).
docker network create grid
docker run -d -p 4442-4444:4442-4444 --net grid --name selenium-hub selenium/hub:4.16.1-20231219
docker run -d --net grid -e SE_EVENT_BUS_HOST=selenium-hub `
--shm-size="2g" `
-e SE_EVENT_BUS_PUBLISH_PORT=4442 `
-e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 `
-e SE_NODE_SESSION_TIMEOUT=60 `
-e SE_ENABLE_TRACING=true `
-e SE_OPTS="--log-level FINE" `
-v ${PWD}/config.toml:/opt/bin/config.toml `
-v ${PWD}/assets:/opt/selenium/assets `
-v /var/run/docker.sock:/var/run/docker.sock `
selenium/node-docker:4.16.1-20231219
Note that these commands reference config.toml from above, and point to a directory called assets that will be filled with reports from the tests. There, you will find new directories created, one for each test run, containing video.mp4 with video recording (if set to true in .side.yml) and sessionCapabilities.json with a report on under which settings the test was executed.
When the grid has started (with the commands above), you can show the Selenium grid UI:http://localhost:4444/ui#
Running your test
From the command line, current directory should be where you have your .side.yml, the config.toml and your test project file, for example, MyTest.side, you can run:
selenium-side-runner .\MyTest.side
If you want to run with parallel execution, for example 5 concurrent tests, use:
selenium-side-runner -w 5 .\MyTest.side
Note! Your project needs to have multiple tests included in a "Test suite" to be executed in parallel. An easy way to do this is to simply duplicate the tests in the project and then add them to the default Test suite. That way, the runner can schedule them at the same time and run them all at once (up to the number of cores in your machine).
Useful links
Main Selenium site: https://www.selenium.dev/