Checker Plugin¶

Checker is a custom pytest plugin that works within the Zelos Test Framework. It's designed to make your tests clean, readable, and easy to write. This documentation will help you get started with using the checker plugin, give example usages, and guide you through its API.

Features¶

Descriptive assertions
Integration with pytest
Customizable operations
Custom configuration

Installation¶

Checker comes pre-installed with the zeloscloud python package. To use it, all you have to do is add into the pytest_plugins list in your conftest.py file..

# You may already have other plugins listed here
pytest_plugins = [
    # Add the checker plugin
    "zeloscloud.pytest.checker",
]

Once this plugin is registered, you'll automatically get a fixture built called check that you can use within your tests. The fixture just simply gets added as an arg to your test, and it will automatically handle building, setup, and teardown within a test function scope.

Example usage:

def test_example(check):
    check.that(1, "is equal to", 1)
    check.that(1, "is less than", 2)
    check.that(1, "is in", [1, 2, 3])

Getting Started with the Checker Plugin¶

You can use any of the pre-registered descriptive operations in checker out of the box.

Pro Tip

You can register custom Checker operations for more expressive tests. See operators for details.

Default Operations¶

The full table of registered operations, their description, and example usage can bee seen below.

Operation	Description	Example Usage
`=`, `==`, `is`, `is equal to`	Checks for equality	`check.that(value, "==", expected)`
`!=`, `is not`, `is not equal to`	Checks for inequality	`check.that(value, "!=", expected)`
`>`, `is greater than`	Checks if a value is greater than another	`check.that(value, ">", comparison_value)`
`>=`, `is greater than or equal to`	Checks if a value is greater than or equal to another	`check.that(value, ">=", comparison_value)`
`<`, `is less than`	Checks if a value is less than another	`check.that(value, "<", comparison_value)`
`<=`, `is less than or equal to`	Checks if a value is less than or equal to another	`check.that(value, "<=", comparison_value)`
`in`, `is in`	Checks if an element is in a collection	`check.that(element, "in", collection)`
`not in`, `is not in`	Checks if an element is not in a collection	`check.that(element, "not in", collection)`
`is close to`, `is around`	Checks if two values are approximately equal (useful for floating-point comparisons)	`check.that(value, "is close to", expected, tolerance=0.01)`
`~=`, `is approximately`, `is approximately equal to`	Checks if two values are approximately equal using pytest approximation	`check.that(value, "~=", expected, abs=0.01)`
`is divisible by`	Checks if one number is divisible by another	`check.that(number, "is divisible by", divisor)`
`contains`	Checks if a collection contains a specific element	`check.that(collection, "contains", element)`
`starts with`	Checks if a string starts with a specified substring	`check.that(string, "starts with", substring)`
`ends with`	Checks if a string ends with a specified substring	`check.that(string, "ends with", substring)`
`is empty`	Checks if a collection or string is empty	`check.that(collection, "is empty")`
`has length`	Checks if a collection or string has a specific length	`check.that(collection, "has length", length)`
`is positive`	Checks if a number is positive	`check.that(number, "is positive")`
`is not positive`	Checks if a number is not positive	`check.that(number, "is not positive")`
`is negative`	Checks if a number is negative	`check.that(number, "is negative")`
`is not negative`	Checks if a number is not negative	`check.that(number, "is not negative")`
`is true`, `is True`	Checks if a boolean value is `True`	`check.that(boolean_value, "is true")`
`is false`, `is False`	Checks if a boolean value is `False`	`check.that(boolean_value, "is false")`
`is instance of`	Checks if an object is an instance of a specific class	`check.that(object, "is instance of", class)`
`has attribute`	Checks if an object has a specific attribute	`check.that(object, "has attribute", attribute_name)`

There can be multiple descriptions for the same operation, and it's purely a preference on which language you want to use to make your tests more readable.

All of these have the same underlying callable operator

check.that(value, "is equal to", expected)
check.that(value, "==", expected)
check.that(value, "=", expected)

Operational Parameters¶

You'll notice some checks may take additional parameters in the form of args/kwargs. These can be passed into the check.that() method and it will pass them along to the operator.

For example, operations like "is close to" and "is approximately can take additional tolerance bounds for more precise checks. These can be passed like such:

check.that(value, "is close to", expected, kwargs={"rel_tol": 0.01})
check.that(value, "is approximately", expected, kwargs={"rel": 0.01})

Duration Checks¶

Duration checks are a feature of the checker plugin that allow you to perform assertions over a duration of time. These checks can run in either blocking or non-blocking mode, making it possible to check conditions that need to be continuously true for a certain period or become true within a specified duration.

These assertions have the ability to run in either blocking or non-blocking modes.

Blocking Mode: In blocking mode, the check will halt execution until the condition becomes true or the specified duration elapses.

Non-Blocking Mode: In non-blocking mode, the check runs in a separate thread, allowing your test to continue executing other tasks.

Note

By default, checks are run in blocking mode. You can pass blocking=False to run these checks in a background thread.

For-Duration Checks¶

For-duration checks are designed to assert that a given condition remains true continuously over a specified period. This can be useful for testing scenarios where stability over time is critical, such as ensuring a sensor reading stays within an acceptable range or a signal maintains a specific value.

Parameters:¶

Parameter	Type	Description	Default
`for_duration_s`	`float`	The total duration in seconds for which the condition should remain true.	`None` (must be provided)
`interval_s`	`float`	The interval in seconds between each condition check.	`0.1`
`blocking`	`bool`	Specifies whether the check should block execution until completion.	`True`

Example usage (blocking):

# Using the default interval of 0.1 seconds
check.that(value, "==", expected, for_duration_s=5.0)

# Specifying an interval of 0.5 seconds
check.that(value, "==", expected, for_duration_s=5.0, interval_s=0.5)

Example usage (non-blocking):

# Using the default interval of 0.1 seconds
check.that(value, "==", expected, for_duration_s=5.0, blocking=False)

# Specifying an interval of 0.5 seconds
check.that(value, "==", expected, for_duration_s=5.0, interval_s=0.5, blocking=False)

# Do other tasks while the check is running in the background
print("Waiting for check to complete...")
time.sleep(6)

Within-Duration Checks¶

Within-duration checks assert that a given condition becomes true within a specified period. This is useful for scenarios where you expect a condition to be met after some delay, such as waiting for a signal to reach a certain state or a sensor to stabilize.

Parameters:¶

Parameter	Type	Description	Default
`within_duration_s`	`float`	The maximum duration in seconds within which the condition should become true.	`None` (must be provided)
`interval_s`	`float`	The interval in seconds between each condition check.	`0.1`
`blocking`	`bool`	Specifies whether the check should block execution until completion.	`True`

Example usage (blocking):

# Using the default interval of 0.1 seconds
check.that(value, "==", expected, within_duration_s=5.0)

# Specifying an interval of 0.5 seconds
check.that(value, "==", expected, within_duration_s=5.0, interval_s=0.5)

Example usage (non-blocking):

# Using the default interval of 0.1 seconds
check.that(value, "==", expected, within_duration_s=5.0, blocking=False)

# Specifying an interval of 0.5 seconds
check.that(value, "==", expected, within_duration_s=5.0, interval_s=0.1, blocking=False)

# Do other tasks while the check is running
print("Waiting for check to complete...")
time.sleep(6)

Checker Output¶

The Checker Plugin provides detailed output during the test execution and a summary at the end in the form of a checkerboard (table of all the checks). This helps you to quickly identify the checks that were performed and their results. Additionally, during the test execution, the plugin logs each check in real-time along with its pass/fail status.

Example Test + Output¶

Given the following test:

def test_various_checks(check):
    # Equality and Inequality
    check.that(10, "==", 10)
    check.that("hello world", "!=", "goodbye")

    # Comparative
    check.that(10, ">", 5)
    check.that(10, "<=", 20)

    # Membership
    check.that(2, "in", [1, 2, 3])
    check.that(4, "not in", [1, 2, 3])

    # String specific
    check.that("hello world", "starts with", "hello")
    check.that("hello world", "ends with", "world")

    # Checking length
    check.that([1, 2, 3], "has length", 3)

    # Approximation
    check.that(3.14159, "is close to", 3.14, rel_tol=0.01)
    check.that(3.14159, "is approximately", 3.14, rel=0.01)

You'll see the following output:

Checkboard — live log checks + final checkerboard output

Example Test with Signals + Output¶

The Checker Plugin also works directly with Zelos Signal types. It will automatically call'.get() to retrieve the latest value before making the assertion. As part of the reporting output, you'll see both the signal name and the value that was used for the check.

Given the following test:

def test_signal_checks(check):
    lhs = ReadSignal(name="lhs", default_value=5)
    rhs = ReadSignal(name="rhs", default_value=10)

    check.that(lhs, "==", 5)
    check.that(rhs, "is equal to", 10)

    check.that(lhs, "<=", rhs)
    check.that(rhs, "is greater than", lhs)

Note

Notice that lhs.get()/rhs.get() are not explicitly called. Checker will call it for you, and extract the name from for reporting

You'll see the following output:

API Reference¶

See zeloscloud.pytest.checker.Checker in the API Reference.