Back to series
· 11 min read

Unit Testing and Test Frameworks


I. What Is Unit Testing?

Unit testing is a software testing method used to verify whether the smallest independently testable code units in a program meet expectations.

Here, a “unit” usually refers to:

  • A function or method.
  • A certain behavior of a class.
  • A standalone module feature.
  • A piece of business logic with clear inputs and outputs.

For example, a function that calculates order discounts:

def calculate_discount(price, rate):
    if price < 0:
        raise ValueError("price cannot be negative")

    if not 0 <= rate <= 1:
        raise ValueError("invalid discount rate")

    return price * (1 - rate)

Multiple unit tests can be written for it:

def test_normal_discount():
    assert calculate_discount(100, 0.2) == 80


def test_zero_discount():
    assert calculate_discount(100, 0) == 100


def test_full_discount():
    assert calculate_discount(100, 1) == 0


def test_negative_price():
    try:
        calculate_discount(-100, 0.2)
    except ValueError:
        return

    raise AssertionError("Expected ValueError")

These tests verify normal discount, zero discount, full discount, and invalid input, respectively.

The core of unit testing is not to verify whether the code can run, but to verify whether the code behaves correctly under different inputs and boundary conditions.

1. Main Goals of Unit Testing

Goal Description
Verify correctness Check whether function output meets expectations
Find defects Detect logic errors and boundary issues as early as possible
Prevent regressions After code changes, confirm existing functionality has not been broken
Clarify behavior Express the expected behavior of functions and modules through tests
Support refactoring Adjust internal implementation while keeping behavior unchanged
Improve design Expose excessive coupling between modules through tests

2. Unit Testing Is Not the Same as Simply Running Code

Ordinary execution mainly focuses on whether the program can run, while unit testing requires explicit verification of results.

For example:

result = calculate_discount(100, 0.2)
print(result)

This code only outputs the result; it does not automatically determine whether the result is correct.

Whereas:

assert calculate_discount(100, 0.2) == 80

will fail directly when the actual result does not meet expectations.

The key to testing is to establish clear expectations and compare the actual result with the expected result.


II. Basic Structure of Unit Testing

A complete unit test usually consists of three stages:

  1. Arrange: Prepare test data and dependencies.
  2. Act: Execute the code under test.
  3. Assert: Verify the execution result.

This structure is also known as the AAA pattern.

def test_calculate_discount():
    # Arrange: prepare data
    price = 100
    rate = 0.2

    # Act: execute the target function
    result = calculate_discount(price, rate)

    # Assert: verify the result
    assert result == 80

1. Arrange: Preparation Stage

Prepare the inputs and environment required for the test.

This includes:

  • Constructing the test object.
  • Preparing input data.
  • Initializing dependencies.
  • Configuring Mock objects.
  • Setting the necessary test state.

The preparation stage should be as simple as possible, avoiding complex operations unrelated to the testing goal.

2. Act: Execution Stage

Call the function, method, or module that needs to be verified.

A test should usually focus on one main behavior and avoid performing many unrelated operations at the same time.

For example:

result = user_service.register("alice", "123456")

3. Assert: Assertion Stage

Compare the actual result with the expected result.

Common assertions include:

assert result == expected
assert result is not None
assert len(result) == 3
assert "admin" in roles

For exceptional behavior, explicit verification is also required:

import pytest

def test_invalid_price():
    with pytest.raises(ValueError):
        calculate_discount(-100, 0.2)

4. Test Execution Flow

flowchart TD
    A["Select test target"] --> B["Prepare test data and dependencies"]
    B --> C["Execute the code under test"]
    C --> D["Obtain actual results"]
    D --> E{"Do the assertions pass?"}
    E -->|"Pass"| F["Record test success"]
    E -->|"Fail"| G["Record failure information"]
    G --> H["Locate the defect"]
    H --> I["Fix the code"]
    I --> C

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style E fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style F fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style G fill:#FBE9E9,stroke:#D9AAAA,color:#334155
    style H fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style I fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

III. What Is a Testing Framework?

A testing framework is a set of software tools used to write, organize, execute, and manage test code.

It typically provides:

  • Test case discovery and execution.
  • Assertion mechanisms.
  • Test environment setup and teardown.
  • Test data management.
  • Test doubles such as mocks and stubs.
  • Test result statistics.
  • Test report generation.
  • Parallel test execution.

Without a testing framework, you can still write tests as ordinary code, but as the number of tests grows, test discovery, execution, result statistics, and environment management become complicated.

A testing framework standardizes these common capabilities, allowing developers to focus on testing business behavior.

1. Core Components of a Testing Framework

Component Purpose
Test Case Define a specific test case
Test Suite Group multiple test cases together
Assertion Determine whether actual results meet expectations
Fixture Manage setup and teardown before and after tests
Test Runner Discover and execute test cases
Mock Simulate the behavior of external dependencies
Reporter Output test results and reports

2. How a Testing Framework Works

flowchart TD
    A["Test Code"] --> B["Test Discovery"]
    B --> C["Build Test Collection"]
    C --> D["Prepare Fixture"]
    D --> E["Execute Test Case"]
    E --> F["Run Assertion"]
    F --> G{"Test Result"}
    G -->|"Passed"| H["Record Pass"]
    G -->|"Failed"| I["Record Fail"]
    H --> J["Clean Up Test Environment"]
    I --> J
    J --> K["Generate Test Report"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style D fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style E fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style G fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style H fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style I fill:#FBE9E9,stroke:#D9AAAA,color:#334155
    style J fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style K fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

Note: Different testing frameworks do not have exactly the same internal implementation. For example, some frameworks organize tests using classes and methods, while others are centered on ordinary functions; the specific mechanisms for fixtures, test discovery, and parallel execution also differ.


4. Common Testing Frameworks

Different programming languages have different testing ecosystems.

1. Python: pytest and unittest

Common Python testing frameworks include pytest and unittest.

pytest

pytest is a widely used third-party testing framework in the Python ecosystem, with concise syntax and strong extensibility.

Installation:

pip install pytest

Test code:

def add(a, b):
    return a + b
def test_add():
    assert add(2, 3) == 5

Run:

pytest

pytest automatically discovers test files and test functions that follow the naming conventions.

Common capabilities:

  • Concise assert assertions.
  • Fixture-based test environment management.
  • Parameterized testing.
  • Mock integration.
  • Plugin extensions.
  • Parallel test support (usually through plugins such as pytest-xdist).

Parameterized test example:

import pytest
@pytest.mark.parametrize(
    "a,b,expected",
    [
        (1, 2, 3),
        (2, 3, 5),
        (10, 20, 30),
    ]
)
def test_add(a, b, expected):
    assert a + b == expected

A single test function can be executed with multiple sets of inputs, reducing duplicated code.

unittest

unittest is a testing framework provided by the Python standard library and requires no additional installation.

import unittest
def add(a, b):
    return a + b
class TestMath(unittest.TestCase):
def test_add(self):
        self.assertEqual(add(2, 3), 5)
def test_positive(self):
        self.assertGreater(add(2, 3), 0)
if __name__ == "__main__":
    unittest.main()

Run:

python -m unittest

Main differences between pytest and unittest:

Comparison pytest unittest
Origin Third-party framework Python standard library
Syntax Concise, function-oriented style Based on test classes
Assertions Use assert directly Use methods such as assertEqual
Fixture Flexible Fixture system setUp, tearDown, etc.
Parameterization Built-in support Usually requires extensions or manual organization
Plugin ecosystem Rich Relatively traditional

2. Java: JUnit

JUnit is a widely used unit testing framework in the Java ecosystem.

Taking JUnit 5 as an example:

import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class CalculatorTest {
@Test
    void testAdd() {
        Calculator calculator = new Calculator();
int result = calculator.add(2, 3);
assertEquals(5, result);
    }
}

Common annotations:

Annotation Purpose
@Test Declares a test method
@BeforeEach Runs before each test method
@AfterEach Runs after each test method
@BeforeAll Runs before all tests start
@AfterAll Runs after all tests end
@ParameterizedTest Declares a parameterized test
@Disabled Temporarily disables a test

The JUnit 5 architecture mainly consists of three parts:

  • JUnit Platform: the test execution platform.
  • JUnit Jupiter: the programming model and extension model of JUnit 5.
  • JUnit Vintage: compatible execution of JUnit 3 and JUnit 4 tests.

JUnit is usually integrated with Maven or Gradle and supports automatically executing tests during the build process.

3. JavaScript / TypeScript: Jest and Vitest

Common testing frameworks for JavaScript and TypeScript projects include Jest and Vitest.

Jest

Jest is a relatively full-featured JavaScript testing framework.

function add(a, b) {
    return a + b;
}
test("add should return the sum", () => {
    expect(add(2, 3)).toBe(5);
});

Run:

npx jest

Common capabilities:

  • Test discovery and execution.
  • Assertions and matchers.
  • Mock functions.
  • Module mocks.
  • Snapshot Testing.
  • Coverage statistics.

Vitest

Vitest is a testing framework for modern JavaScript toolchains, closely integrated with the Vite ecosystem.

import { describe, it, expect } from "vitest";
function add(a, b) {
    return a + b;
}
describe("add", () => {
    it("should return the sum", () => {
        expect(add(2, 3)).toBe(5);
    });
});

Run:

npx vitest

Both Jest and Vitest support common testing capabilities, but Vitest usually has more natural configuration integration in Vite projects.

4. Go: testing

Go has a built-in testing package, so basic unit tests can be written without installing a third-party testing framework.

Production code:

package calculator
func Add(a, b int) int {
    return a + b
}

Test code:

package calculator
import "testing"
func TestAdd(t *testing.T) {
    result := Add(2, 3)
if result != 5 {
        t.Errorf("expected 5, got %d", result)
    }
}

File naming typically uses:

calculator.go
calculator_test.go

Run:

go test ./...

Go's testing system also supports:

  • Table-driven Tests.
  • Subtests.
  • Benchmarks.
  • Fuzzing.
  • Example Tests.

Table-driven tests:

func TestAdd(t *testing.T) {
    cases := []struct {
        a, b, expected int
    }{
        {1, 2, 3},
        {2, 3, 5},
        {10, 20, 30},
    }
for _, tc := range cases {
        result := Add(tc.a, tc.b)
if result != tc.expected {
            t.Errorf(
                "Add(%d, %d) = %d, want %d",
                tc.a, tc.b, result, tc.expected,
            )
        }
    }
}

5. Rust: Built-in Testing Mechanism

Rust provides a built-in testing mechanism, marking test functions with #[test].

fn add(a: i32, b: i32) -> i32 {
    a + b
}
#[test]
fn test_add() {
    assert_eq!(add(2, 3), 5);
}

Run:

cargo test

Rust also supports:

  • assert!: verifies a boolean condition.
  • assert_eq!: verifies equality.
  • assert_ne!: verifies inequality.
  • #[should_panic]: verifies whether a test panics as expected.
  • #[cfg(test)]: controls the compilation conditions for test code.

Example:

#[test]
#[should_panic]
fn test_invalid_operation() {
    panic!("expected panic");
}

Rust's testing mechanism is deeply integrated with Cargo, making it suitable for incorporating tests into the project build process.


V. Fixture: Test Environment Management

A Fixture is a test resource management mechanism provided by testing frameworks, used to prepare the environment before test execution and clean up after tests finish.

For example, a user service may depend on:

  • Database connections.
  • Test users.
  • Temporary files.
  • Mock objects.
  • Test configuration.

If every test repeatedly prepares these resources, a large amount of duplicated code will result.

Fixtures can manage these preparation and cleanup operations in a unified way.

1. pytest Fixture Example

import pytest
@pytest.fixture
def user():
    return {
        "id": 1,
        "name": "Alice"
    }
def test_user_name(user):
    assert user["name"] == "Alice"

pytest automatically passes the Fixture to the test function as an argument.

2. Fixture Scopes

pytest provides different Fixture lifecycles:

Scope Lifecycle
function Each test function
class Each test class
module Each test module
package Each test package
session The entire test session

For example:

@pytest.fixture(scope="session")
def database_connection():
    connection = create_connection()
yield connection
connection.close()

Here, yield separates resource preparation from resource cleanup.

3. Fixture Lifecycle

flowchart TD
    A["Start tests"] --> B["Initialize Fixture"]
    B --> C["Prepare test resources"]
    C --> D["Execute test cases"]
    D --> E["Collect test results"]
    E --> F{"Are there more tests?"}
    F -->|"Yes"| D
    F -->|"No"| G["Execute cleanup logic"]
    G --> H["Release resources"]
    H --> I["End tests"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style G fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style H fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style I fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

The larger the Fixture scope, the higher the degree of resource reuse usually is, but the risk of sharing state between tests also increases.

For example, if multiple tests share the same database record and one test modifies the data, it may affect other tests.

Therefore, Fixture design needs to balance resource creation cost and test isolation.


VI. Mock, Stub, and Test Doubles

In real development, a function may depend on databases, network services, message queues, or third-party APIs.

If unit tests directly call these real dependencies, several problems arise:

  • Test execution is slower.
  • Tests depend on external networks and service states.
  • Test data is difficult to control.
  • External service failures may cause test failures.
  • Test results are unstable.

Therefore, unit tests often use Test Doubles to isolate external dependencies.

1. Common Types of Test Doubles

Type Purpose
Dummy A placeholder object used only to satisfy parameter requirements
Stub Provides preset return results
Mock Verifies whether dependencies are called as expected
Fake Provides a working simplified implementation
Spy Records real call processes for later verification

These concepts describe different Test Double roles. In actual tools, an object may have multiple characteristics at the same time.

2. Stub Example

Suppose an order service needs to call a payment service:

class PaymentService:
    def pay(self, amount):
        raise NotImplementedError
class OrderService:
    def __init__(self, payment_service):
        self.payment_service = payment_service
def create_order(self, amount):
        return self.payment_service.pay(amount)

During testing, a Stub can be used to simulate the payment result:

class FakePaymentService:
    def pay(self, amount):
        return {
            "success": True,
            "amount": amount
        }
def test_create_order():
    payment = FakePaymentService()
    order = OrderService(payment)
result = order.create_order(100)
assert result["success"] is True
    assert result["amount"] == 100

Here, a simplified payment implementation is used to avoid calling the real payment system.

3. Mock Example

If you want to verify whether the payment service is called correctly, you can use a Mock:

from unittest.mock import Mock
def test_create_order_calls_payment():
    payment = Mock()
    payment.pay.return_value = {
        "success": True
    }
order = OrderService(payment)
    result = order.create_order(100)
assert result["success"] is True
    payment.pay.assert_called_once_with(100)

This test not only verifies the return result but also verifies whether the order service calls the payment service with the correct arguments.

4. Boundaries of Mock Usage

More Mocks are not necessarily better.

Excessive mocking may lead to:

  • Tests only verify implementation details rather than business behavior.
  • A large number of tests need to be modified after code refactoring.
  • Tests pass, but integration with real dependencies still fails.
  • Test code is highly coupled to the internal structure of production code.

An important principle is:

Prioritize verifying externally observable behavior rather than over-verifying internal call processes.

For complex database interactions, message queues, and third-party APIs, integration tests are also needed to verify real collaboration behavior.


VII. Unit Test Coverage

Test Coverage is used to measure the degree to which code is covered during test execution.

Common metrics include:

1. Statement Coverage

Measures how many executable statements have been executed by tests.

For example:

def calculate(value):
    if value > 0:
        return value * 2
return 0

If only this is tested:

calculate(10)

then the statement corresponding to return 0 is not executed.

Statement coverage cannot prove that all branches have been tested.

2. Branch Coverage

Measures whether all conditional branches in the program have been executed.

For example:

if value > 0:
    return "positive"
else:
    return "non-positive"

At least the following need to be tested:

  • value > 0 is true.
  • value > 0 is false.

Branch coverage better reflects the testing of conditional judgments than statement coverage alone.

3. Condition Coverage

For each boolean subexpression in a compound condition, checks whether both its true and false outcomes have been tested.

For example:

if age >= 18 and has_permission:
    allow()

This contains two conditions:

  • age >= 18
  • has_permission

Condition coverage focuses on whether each of the two conditions has been evaluated as both True and False.

Note that condition coverage and branch coverage are not the same.

4. Path Coverage

Measures whether different execution paths in the program are covered.

For programs with multiple conditions and loops, the number of paths may be very large, and may even be difficult to exhaust as the number of loop iterations increases.

Therefore, complete path coverage is usually unrealistic in complex programs.

5. Comparison of Coverage Metrics

Metric What It Checks Limitation
Statement coverage Whether statements are executed Does not guarantee complete branch coverage
Branch coverage Whether decision branches are executed Does not guarantee all condition combinations
Condition coverage True/false outcomes of subconditions Does not necessarily cover all branch combinations
Path coverage Execution paths Difficult to exhaust in complex programs
Function coverage Whether functions are called Does not mean the function behaves correctly

6. How to Understand Coverage?

Suppose a project's unit test coverage reaches 90%; this does not mean the code is 90% correct.

Coverage can only indicate how much code the test execution touched; it cannot directly prove whether the assertions are sufficient.

For example:

def divide(a, b):
    return a / b