Zelos Test Framework¶
The Zelos Test Framework is a set of libraries for interacting with hardware, writing automated tests using Python, and generating automatic reports.
Pro Tip
See Installing Zelos for detailed instructions to get the Zelos Test Framework installed on your computer.
Basic Concepts¶
The Zelos Test Framework is tightly integrated with pytest, a popular Python test framework. It extends the core pytest concepts with additional functionality for flashing embededed devices, talking to test equipment, inspecting communication, and generating reports.
Project Structure¶
There are several important files required to setup a Zelos Test project. If using the zelos create project command as described in Creating a New Project, you will automatically get a folder structure with defaults setup.
new-project --> the name of your project
|-- tests --> all your tests live here
| |-- test_checker.py --> example test showcasing the checker
| `-- ...
|-- confest.py --> defines fixtures, plugins, helpers for your tests
|-- pytest.ini --> configures pytest, includes a basic logging config
`-- README.md --> contains some helper text about the project
Writing a Test¶
Tests are written as pytest functions that start with test_. Required arguments are automatically supplied via pytest
fixtures.
Warning
Test functions and files must start with test_ otherwise they will be ignored by test collection.
# This defines a test called test_checker. The arg "check" is a pytest fixture
# that gets automatically populated when the test framework calls the test.
def test_checker(check):
# Using the check plugin, we can check various conditions without causing
# our test to fail instantly. They are summarized in a table format at the
# end of each test, showing the pass / fail status for each check.
check.that(10, "==", 10)
check.that("hello world", "!=", "goodbye")
check.that(10, ">", 5)
check.that(10, "<=", 20)
check.that(2, "in", [1, 2, 3])
check.that(4, "not in", [1, 2, 3])
check.that("hello world", "starts with", "hello")
check.that("hello world", "ends with", "world")
check.that([1, 2, 3], "has length", 3)
Running a Test¶
The command pytest can be used to run a suite of tests. This example runs the test_checker example from above. As
the test runs, it logs the check evaluation in real-time, as well as printing the Zelos Checkerboard summarizing the
test result.
$ pytest
============================= test session starts ==============================
...
collected 1 item
tests/test_checker.py::test_checker
-------------------------------- live log call ---------------------------------
INFO zelos-checker:items.py:86 10 == 10 --> PASSED
INFO zelos-checker:items.py:86 hello world != goodbye --> PASSED
INFO zelos-checker:items.py:86 10 > 5 --> PASSED
INFO zelos-checker:items.py:86 10 <= 20 --> PASSED
INFO zelos-checker:items.py:86 2 in [1, 2, 3] --> PASSED
INFO zelos-checker:items.py:86 4 not in [1, 2, 3] --> PASSED
INFO zelos-checker:items.py:86 hello world starts with hello --> PASSED
INFO zelos-checker:items.py:86 hello world ends with world --> PASSED
INFO zelos-checker:items.py:86 [1, 2, 3] has length 3 --> PASSED
PASSED
Zelos Checkerboard
test_checker
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┓
┃ timestamp ┃ lhs ┃ operator ┃ rhs ┃ parameters ┃ result ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━┩
│ 2024-04-02T… │ 10 │ == │ 10 │ │ PASSED │
│ 2024-04-02T… │ hello world │ != │ goodbye │ │ PASSED │
│ 2024-04-02T… │ 10 │ > │ 5 │ │ PASSED │
│ 2024-04-02T… │ 10 │ <= │ 20 │ │ PASSED │
│ 2024-04-02T… │ 2 │ in │ [1, 2, 3] │ │ PASSED │
│ 2024-04-02T… │ 4 │ not in │ [1, 2, 3] │ │ PASSED │
│ 2024-04-02T… │ hello world │ starts with │ hello │ │ PASSED │
│ 2024-04-02T… │ hello world │ ends with │ world │ │ PASSED │
│ 2024-04-02T… │ [1, 2, 3] │ has length │ 3 │ │ PASSED │
└──────────────┴─────────────┴──────────────┴───────────┴─────────────┴────────┘
============================== 1 passed in 0.01s ===============================