使用自定义标记

这里有一些使用 如何使用属性标记测试函数 机制的示例。

标记测试函数并选择它们运行

你可以像这样使用自定义元数据“标记”一个测试函数

# content of test_server.py

import pytest


@pytest.mark.webtest
def test_send_http():
    pass  # perform some webtest test for your app


@pytest.mark.device(serial="123")
def test_something_quick():
    pass


@pytest.mark.device(serial="abc")
def test_another():
    pass


class TestClass:
    def test_method(self):
        pass

然后你可以限制测试运行，只运行标记为 webtest 的测试

$ pytest -v -m webtest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 3 deselected / 1 selected

test_server.py::test_send_http PASSED                                [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

或者相反，运行除 webtest 测试之外的所有测试

$ pytest -v -m "not webtest"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 1 deselected / 3 selected

test_server.py::test_something_quick PASSED                          [ 33%]
test_server.py::test_another PASSED                                  [ 66%]
test_server.py::TestClass::test_method PASSED                        [100%]

===================== 3 passed, 1 deselected in 0.12s ======================

此外，你可以限制测试运行，只运行匹配一个或多个标记关键字参数的测试，例如，只运行标记为 device 且具有特定 serial="123" 的测试

$ pytest -v -m "device(serial='123')"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 3 deselected / 1 selected

test_server.py::test_something_quick PASSED                          [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

注意

标记表达式中仅支持关键字参数匹配。

注意

标记表达式中仅支持 int, (未转义的) str, bool 和 None 值。

基于节点 ID 选择测试

你可以提供一个或多个 节点 ID 作为位置参数，以仅选择指定的测试。这使得根据模块、类、方法或函数名称选择测试变得容易。

$ pytest -v test_server.py::TestClass::test_method
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 1 item

test_server.py::TestClass::test_method PASSED                        [100%]

============================ 1 passed in 0.12s =============================

你也可以按类选择

$ pytest -v test_server.py::TestClass
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 1 item

test_server.py::TestClass::test_method PASSED                        [100%]

============================ 1 passed in 0.12s =============================

或者选择多个节点

$ pytest -v test_server.py::TestClass test_server.py::test_send_http
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 2 items

test_server.py::TestClass::test_method PASSED                        [ 50%]
test_server.py::test_send_http PASSED                                [100%]

============================ 2 passed in 0.12s =============================

注意

节点 ID 的形式为 module.py::class::method 或 module.py::function。节点 ID 控制收集哪些测试，因此 module.py::class 将选择类上的所有测试方法。节点也为参数化的 fixture（固件）或测试的每个参数创建，因此选择参数化测试必须包含参数值，例如 module.py::function[param]。

当使用 -rf 选项运行 pytest 时，失败测试的节点 ID 会显示在测试摘要信息中。你也可以从 pytest --collect-only 的输出中构造节点 ID。

使用 -k expr 基于名称选择测试

在版本 2.0/2.3.4 中添加。

你可以使用 -k 命令行选项来指定一个表达式，该表达式对测试名称实现子字符串匹配，而不是 -m 提供的对标记的精确匹配。这使得根据名称选择测试变得容易

在版本 5.4 中更改。

表达式匹配现在不区分大小写。

$ pytest -v -k http  # running with the above defined example module
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 3 deselected / 1 selected

test_server.py::test_send_http PASSED                                [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

你也可以运行除与关键字匹配的测试之外的所有测试

$ pytest -k "not send_http" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 1 deselected / 3 selected

test_server.py::test_something_quick PASSED                          [ 33%]
test_server.py::test_another PASSED                                  [ 66%]
test_server.py::TestClass::test_method PASSED                        [100%]

===================== 3 passed, 1 deselected in 0.12s ======================

或者选择 “http” 和 “quick” 测试

$ pytest -k "http or quick" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 4 items / 2 deselected / 2 selected

test_server.py::test_send_http PASSED                                [ 50%]
test_server.py::test_something_quick PASSED                          [100%]

===================== 2 passed, 2 deselected in 0.12s ======================

你可以使用 and、or、not 和括号。

除了测试的名称之外，-k 还匹配测试父级的名称（通常是文件和它所在的类的名称）、在测试函数上设置的属性、应用于它或其父级的标记以及显式添加到它或其父级的任何 extra keywords。

注册标记

为你的测试套件注册标记很简单

# content of pytest.ini
[pytest]
markers =
    webtest: mark a test as a webtest.
    slow: mark test as slow.

可以注册多个自定义标记，方法是在其自己的行中定义每个标记，如上面的示例所示。

你可以询问你的测试套件存在哪些标记 - 该列表包括我们刚刚定义的 webtest 和 slow 标记

$ pytest --markers
@pytest.mark.webtest: mark a test as a webtest.

@pytest.mark.slow: mark test as slow.

@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://pytest.pythonlang.cn/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings

@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.

@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://pytest.pythonlang.cn/en/stable/reference/reference.html#pytest-mark-skipif

@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://pytest.pythonlang.cn/en/stable/reference/reference.html#pytest-mark-xfail

@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://pytest.pythonlang.cn/en/stable/how-to/parametrize.html for more info and examples.

@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://pytest.pythonlang.cn/en/stable/explanation/fixtures.html#usefixtures

@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.

@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.

有关如何从插件添加和使用标记的示例，请参阅 自定义标记和用于控制测试运行的命令行选项。

注意

建议显式注册标记，以便

• 在你的测试套件中有一个地方定义你的标记

• 通过 pytest --markers 请求现有标记会给出良好的输出

• 如果使用 --strict-markers 选项，则函数标记中的拼写错误将被视为错误。

标记整个类或模块

你可以将 pytest.mark 装饰器与类一起使用，以将标记应用于其所有测试方法

# content of test_mark_classlevel.py
import pytest


@pytest.mark.webtest
class TestClass:
    def test_startup(self):
        pass

    def test_startup_and_more(self):
        pass

这等效于直接将装饰器应用于两个测试函数。

要在模块级别应用标记，请使用 pytestmark 全局变量

import pytest
pytestmark = pytest.mark.webtest

或多个标记

pytestmark = [pytest.mark.webtest, pytest.mark.slowtest]

由于历史原因，在引入类装饰器之前，可以在测试类上像这样设置 pytestmark 属性

import pytest


class TestClass:
    pytestmark = pytest.mark.webtest

使用参数化时标记单个测试

当使用参数化时，应用标记将使其应用于每个单独的测试。但是，也可以将标记应用于单个测试实例

import pytest


@pytest.mark.foo
@pytest.mark.parametrize(
    ("n", "expected"), [(1, 2), pytest.param(1, 3, marks=pytest.mark.bar), (2, 3)]
)
def test_increment(n, expected):
    assert n + 1 == expected

在本例中，“foo” 标记将应用于三个测试中的每一个，而 “bar” 标记仅应用于第二个测试。Skip 和 xfail 标记也可以以这种方式应用，请参阅 使用参数化进行 Skip/xfail。

自定义标记和用于控制测试运行的命令行选项

插件可以提供自定义标记并基于它实现特定行为。这是一个自包含的示例，它添加了一个命令行选项和一个参数化的测试函数标记，以运行通过命名环境指定的测试

# content of conftest.py

import pytest


def pytest_addoption(parser):
    parser.addoption(
        "-E",
        action="store",
        metavar="NAME",
        help="only run tests matching the environment NAME.",
    )


def pytest_configure(config):
    # register an additional marker
    config.addinivalue_line(
        "markers", "env(name): mark test to run only on named environment"
    )


def pytest_runtest_setup(item):
    envnames = [mark.args[0] for mark in item.iter_markers(name="env")]
    if envnames:
        if item.config.getoption("-E") not in envnames:
            pytest.skip(f"test requires env in {envnames!r}")

使用此本地插件的测试文件

# content of test_someenv.py

import pytest


@pytest.mark.env("stage1")
def test_basic_db_operation():
    pass

以及一个示例调用，指定了与测试需要的环境不同的环境

$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item

test_someenv.py s                                                    [100%]

============================ 1 skipped in 0.12s ============================

这是另一个示例，它精确地指定了需要的环境

$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item

test_someenv.py .                                                    [100%]

============================ 1 passed in 0.12s =============================

--markers 选项始终为你提供可用标记的列表

$ pytest --markers
@pytest.mark.env(name): mark test to run only on named environment

@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://pytest.pythonlang.cn/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings

@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.

@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://pytest.pythonlang.cn/en/stable/reference/reference.html#pytest-mark-skipif

@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://pytest.pythonlang.cn/en/stable/reference/reference.html#pytest-mark-xfail

@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://pytest.pythonlang.cn/en/stable/how-to/parametrize.html for more info and examples.

@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://pytest.pythonlang.cn/en/stable/explanation/fixtures.html#usefixtures

@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.

@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.

将可调用对象传递给自定义标记

下面是将在接下来的示例中使用的配置文件

# content of conftest.py
import sys


def pytest_runtest_setup(item):
    for marker in item.iter_markers(name="my_marker"):
        print(marker)
        sys.stdout.flush()

自定义标记可以设置其参数，即 args 和 kwargs 属性，可以通过将其作为可调用对象调用或使用 pytest.mark.MARKER_NAME.with_args 来定义。这两种方法在大多数情况下都达到相同的效果。

但是，如果存在一个可调用对象作为没有关键字参数的单个位置参数，则使用 pytest.mark.MARKER_NAME(c) 不会将 c 作为位置参数传递，而是使用自定义标记装饰 c（请参阅 MarkDecorator）。幸运的是，pytest.mark.MARKER_NAME.with_args 提供了帮助

# content of test_custom_marker.py
import pytest


def hello_world(*args, **kwargs):
    return "Hello World"


@pytest.mark.my_marker.with_args(hello_world)
def test_with_args():
    pass

输出如下

$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef0001>,), kwargs={})
.
1 passed in 0.12s

我们可以看到自定义标记的参数集已扩展为函数 hello_world。这是将自定义标记创建为可调用对象（在幕后调用 __call__）和使用 with_args 之间的关键区别。

读取从多个位置设置的标记

如果你在测试套件中大量使用标记，你可能会遇到一个标记多次应用于一个测试函数的情况。从插件代码中，你可以读取所有此类设置。示例

# content of test_mark_three_times.py
import pytest

pytestmark = pytest.mark.glob("module", x=1)


@pytest.mark.glob("class", x=2)
class TestClass:
    @pytest.mark.glob("function", x=3)
    def test_something(self):
        pass

在这里，我们有标记 “glob” 应用于同一个测试函数三次。从 conftest 文件中，我们可以像这样读取它

# content of conftest.py
import sys


def pytest_runtest_setup(item):
    for mark in item.iter_markers(name="glob"):
        print(f"glob args={mark.args} kwargs={mark.kwargs}")
        sys.stdout.flush()

让我们在不捕获输出的情况下运行它，看看我们得到什么

$ pytest -q -s
glob args=('function',) kwargs={'x': 3}
glob args=('class',) kwargs={'x': 2}
glob args=('module',) kwargs={'x': 1}
.
1 passed in 0.12s

使用 pytest 标记特定于平台的测试

假设你有一个测试套件，它为特定平台标记测试，即 pytest.mark.darwin、pytest.mark.win32 等，并且你还有在所有平台上运行且没有特定标记的测试。如果你现在想要一种仅运行针对你的特定平台的测试的方法，你可以使用以下插件

# content of conftest.py
#
import sys

import pytest

ALL = set("darwin linux win32".split())


def pytest_runtest_setup(item):
    supported_platforms = ALL.intersection(mark.name for mark in item.iter_markers())
    plat = sys.platform
    if supported_platforms and plat not in supported_platforms:
        pytest.skip(f"cannot run on platform {plat}")

然后，如果测试是为不同的平台指定的，则会跳过这些测试。让我们做一个小的测试文件来展示它的样子

# content of test_plat.py

import pytest


@pytest.mark.darwin
def test_if_apple_is_evil():
    pass


@pytest.mark.linux
def test_if_linux_works():
    pass


@pytest.mark.win32
def test_if_win32_crashes():
    pass


def test_runs_everywhere():
    pass

然后你将看到两个测试被跳过，两个测试按预期执行

$ pytest -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items

test_plat.py s.s.                                                    [100%]

========================= short test summary info ==========================
SKIPPED [2] conftest.py:13: cannot run on platform linux
======================= 2 passed, 2 skipped in 0.12s =======================

请注意，如果你像这样通过标记命令行选项指定平台

$ pytest -m linux
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 3 deselected / 1 selected

test_plat.py .                                                       [100%]

===================== 1 passed, 3 deselected in 0.12s ======================

那么未标记的测试将不会运行。因此，这是一种将运行限制为特定测试的方法。

基于测试名称自动添加标记

如果你的测试套件中测试函数名称指示某种类型的测试，你可以实现一个 hook，自动定义标记，以便你可以将 -m 选项与它一起使用。让我们看看这个测试模块

# content of test_module.py


def test_interface_simple():
    assert 0


def test_interface_complex():
    assert 0


def test_event_simple():
    assert 0


def test_something_else():
    assert 0

我们想要动态定义两个标记，并且可以在 conftest.py 插件中完成它

# content of conftest.py

import pytest


def pytest_collection_modifyitems(items):
    for item in items:
        if "interface" in item.nodeid:
            item.add_marker(pytest.mark.interface)
        elif "event" in item.nodeid:
            item.add_marker(pytest.mark.event)

我们现在可以使用 -m option 来选择一个集合

$ pytest -m interface --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 2 deselected / 2 selected

test_module.py FF                                                    [100%]

================================= FAILURES =================================
__________________________ test_interface_simple ___________________________
test_module.py:4: in test_interface_simple
    assert 0
E   assert 0
__________________________ test_interface_complex __________________________
test_module.py:8: in test_interface_complex
    assert 0
E   assert 0
========================= short test summary info ==========================
FAILED test_module.py::test_interface_simple - assert 0
FAILED test_module.py::test_interface_complex - assert 0
===================== 2 failed, 2 deselected in 0.12s ======================

或选择 “event” 和 “interface” 测试

$ pytest -m "interface or event" --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 1 deselected / 3 selected

test_module.py FFF                                                   [100%]

================================= FAILURES =================================
__________________________ test_interface_simple ___________________________
test_module.py:4: in test_interface_simple
    assert 0
E   assert 0
__________________________ test_interface_complex __________________________
test_module.py:8: in test_interface_complex
    assert 0
E   assert 0
____________________________ test_event_simple _____________________________
test_module.py:12: in test_event_simple
    assert 0
E   assert 0
========================= short test summary info ==========================
FAILED test_module.py::test_interface_simple - assert 0
FAILED test_module.py::test_interface_complex - assert 0
FAILED test_module.py::test_event_simple - assert 0
===================== 3 failed, 1 deselected in 0.12s ======================