This article describes how to unit test in python. I learned that once an application grows over 150 lines it becomes harder and harder to understand without testing individual smaller functions. The solution should be as simple as possible and described in a well maintained resource. Therefore I selected the standard pytest library.¹ This article assumes you understand what testing is, the python documentation on unit testing & testing with pytest.^{2 3} For this howto we assume you have a sample project with a module my_sum_module ⁴ and a script sum_script.py. We add now unit tests for the script and the module.

project
├── my_sum_module
│   ├── __init__.py
│   ├── precise_sum.py
│   └── rounded_sum.py
└── sum_script.py

Setup the Test Structure

Create a tests folder in you project. Add a test file test_{test_suite_name}.py into the folder.

mkdir myproject/tests
touch myproject/tests/test_{test_suites}.py

Write Unit Tests

To write test install pytest with pipenv install pytest and import the unit test library & runner into test_{test_suite_name}.py. Then import the module that contains the code. Then write your test cases as functions. Use standard assert statements that evaluate to True or False.

import pytest

    
# Add test functions. 
def test_sum():
	data = [1, 2, 3]        
	result = sum(data)        
	# Write your assertions
	assert result == 6
	
def test_sum_tuple():        
	assert sum((1, 2, 2)) == 6

Structure the test in the following workflow:

1. Arrange, or set up, the conditions for the test
2. Act by calling some function or method
3. Assert that some end condition is true

Make sure tests are repeatable and run your test multiple times to make sure it gives the same result every time. Ensure there are no side effect (db changes, filesystem, attribute of class). Refactor the code if you have to many side effects or break the Single Responsibility Principle. There are some simple techniques you can use to test parts of your application that have many side effects:

• Refactoring code to follow the Single Responsibility Principle
• Mocking out any method or function calls to remove side effects ⁵
• Using integration testing instead of unit testing for this piece of the application

Manage Input Data with Fixtures

Setup pieces of data or test doubles for some of the objects in your code with fixtures. Fixtures model the dependencies explicitly as function and as inputs to the test cases. Further they can depend on each other and improve modularity / DRY.

@pytest.fixture
def example_numbers():
	return [ 1, 2, 3] 

def test_sum(example_numbers):      
	result = sum(example_numbers)        
	assert result == 6

When several tests use the same underlying test data use a fixture by pulling into a function annotated with @pytest.fixture. Name your fixture something specific, so you can quickly determine if you want to use it when writing new tests in the future! Be careful with fixture for tests that require slight variations in the data. Littering your test suite with fixtures is worse than littering it with plain data or objects because of the added layer of indirection.

To further reuse fixtures across modules move fixtures from test modules into more general fixture-related modules. Import them back into any test modules that need them. pytest looks for conftest.py modules throughout the directory structure. Each conftest.py provides configuration for the file tree pytest finds it in. You can use any fixtures that are defined in a particular conftest.py throughout the file’s parent directory and in any subdirectories.

Execute and Evaluate Tests

pipenv run python -m pytest

You can add plugins like pytest-cov and then add attributes that show you several types of coverages.

Or you use vscode - shift cmd p -> "Debug All Unit Tests".

It’s not always as easy as creating a static value for the input like a string or a number. Sometimes, your application will require an instance of a class or a context. The data that you create as an input is known as a fixture. It’s widespread practice to create fixtures and reuse them. If you’re running the same test, passing different values each time, and expecting the same result, this is known as parameterization.

Mock External libs/apis

• Practical intro into how to use the mock library: Getting Started with Mocking in Python - Semaphore
• Understand how the mock works and the traps:
  • https://changhsinlee.com/pytest-mock/
  • https://medium.com/uckey/how-mock-patch-decorator-works-in-python-37acd8b78ae

Measure Test Coverage

Use pytest with pytest-cov. You measure code coverage in multiple ways and set the goal it at 100% - and exclude lines that can't be tested (with an annotation). ⁶

pipenv install pytest-cov
pipenv run python -m pytest --cov

Write Integration Test

To integration test an application act like a consumer or user: Calling an HTTP REST API, calling a Python API, calling a web service or running a command line.

Write integration tests in the same way as a unit test, following the Input, Execute, and Assert pattern. Integration tests check more components at once and therefore will have more side effects than a unit test. Also, you need to setup more fixtures to be in place, like a database, a network socket, or a configuration file for integration tests. Therefore, integration tests take longer to execute. Separate your unit tests and your integration tests. So, you may only want to run integration tests before you push to production instead of once on every commit. Set them up in different folders:

project/
│
├── my_app/
│   └── __init__.py
│
└── tests/
|
├── unit/
|   ├── __init__.py
|   └── test_sum.py
|
└── integration/
|
├── fixtures/
|   ├── test_basic.json
|   └── test_complex.json
├── __init__.py
└── test_integration.py

pipenv run python -m pytest tests/integration

Many integration tests will require backend data like a database to exist with certain values. These types of integration tests will depend on different test fixtures to make sure they are repeatable and predictable. Store the test data in a folder within your integration testing folder called fixtures to indicate that it contains test data. Then, within your tests, you can load the data and run the test.

# define two different fixtures test_basic, test_complex

def test_customer_count(test_basic):
	app = App(database=test_basic)
	assert len(self.app.customers) == 100    

def test_existence_of_customer(test_basic):
	app = App(database=test_basic)
	customer = app.get_customer(id=10)        
	assert customer.name == "Org XYZ"        
	assert customer.address == "10 Red Road, Reading"    

def test_customer_count(test_complex):
	app = App(database=test_complex)
	assert len(self.app.customers) = 10000    
	
def test_existence_of_customer(test_complex):
	app = App(database=test_complex)
	customer = app.get_customer(id=9999)        
	assert customer.name = u"バナナ"        
	assert customer.address == "10 Red Road, Akihabara, Tokyo"

If your application depends on data from a remote location, like a remote API, store remote fixtures locally so they can be recalled and sent to the application. E.g. the requests library has a complimentary package called responses that gives you ways to create response fixtures and save them in your test folders.

Keeping Your Test Code Clean

Follow the DRY principle when writing tests: Don’t Repeat Yourself. Use Test fixtures and functions to produce easier to maintain test code.

Other Topics

You can also test in different virtual environments, linting code with flake8, test performance with timeit or pytest-benchmark and security with bandit.³

Django and flask provide their own implementation based on unittest to make the setup simpler.³

Best Pratices

Be efficient & effective: Take a risk-based testing approach and start with testing the most critical functionality.⁷ Do "local" integration test and only if required use mocking. Learn about good and bad tests (https://late.am/post/2015/04/20/good-test-bad-test.html)

Read and apply the list with general testing rules. E.g. run unit test continously while coding.

Use TDD ⁸

Footnotes & Resources

• A greate intro into the different types of python testing: https://miguelgfierro.com/blog/2018/a-beginners-guide-to-python-testing/
• Use continous code quality & security: https://sonarcloud.io/
• Overview on testing: https://www.fullstackpython.com/testing.html
• Tool to automatically generate unit tests: https://www.ponicode.com