[DONT_REVIEW] Unified config declaration: Parser.addconfig and declarative fallbacks

changelog/10551.breaking.rst

@@ -0,0 +1 @@
+An empty value explicitly set for :confval:`log_cli_format`, :confval:`log_cli_date_format`, :confval:`log_file_format` or :confval:`log_file_date_format` is now used as configured. Previously it was treated as unset and the corresponding :confval:`log_format` or :confval:`log_date_format` was used instead.

changelog/10551.deprecation.rst

@@ -0,0 +1,3 @@
+Reading or writing a configuration setting as ``config.option.<name>`` is deprecated; read it with :func:`config.getini <pytest.Config.getini>` or :class:`config.settings <pytest.Settings>`. This affects only settings declared with :func:`parser.addconfig <pytest.Parser.addconfig>` or with a command line option that overrides a configuration option -- plain :func:`parser.addoption <pytest.Parser.addoption>` options are unaffected.
+
+Reading one used to be a trap: the attribute was ``None`` whenever the command line option was absent, even for a setting the configuration file set. It now holds the resolved value.

changelog/10551.feature.1.rst

@@ -0,0 +1,8 @@
+:func:`parser.addini <pytest.Parser.addini>` now accepts a ``fallback`` argument, naming another registered configuration option to take the value from when this one is not set:
+
+.. code-block:: python
+
+    parser.addini("log_format", "Log format", type=str, default=DEFAULT)
+    parser.addini("log_cli_format", "Live log format", type=str, fallback="log_format")
+
+A fallback must already be registered and must have the same type. Several may be given, and the first one that is configured wins.

changelog/10551.feature.2.rst

@@ -0,0 +1,14 @@
+New :func:`parser.addconfig <pytest.Parser.addconfig>`, which registers a configuration option together with a command line option that sets it, in one call and with one type:
+
+.. code-block:: python
+
+    parser.addconfig(
+        "log_file_mode",
+        "Log file open mode",
+        type=Literal["w", "a"],
+        default="w",
+        cli="--log-file-mode",
+        group="logging",
+    )
+
+The command line option overrides the configuration option rather than being a separate value, so :func:`config.getini <pytest.Config.getini>` reads the setting whichever way the user supplied it. The value is also available as ``config.option.<name>``, and the type is enforced for all sources.

changelog/10551.feature.3.rst

@@ -0,0 +1,11 @@
+New :class:`config.settings <pytest.Settings>`, a mapping of every configuration setting to its resolved value:
+
+.. code-block:: python
+
+    config.settings["log_level"]  # the value, whatever set it
+    config.settings.source_of("log_level")  # where it came from
+    config.settings.spec("log_level")  # how it was declared
+
+It answers for a setting whichever source supplied it -- a configuration file, an ``-o`` override, or a command line option declared for it -- and :func:`config.getini <pytest.Config.getini>` is now a thin adapter over it. Command line options that are not configuration settings live in ``config.settings.options``, keyed by their argparse ``dest``.
+
+This API is experimental; its behaviour may change in future releases.

changelog/10551.improvement.1.rst

@@ -0,0 +1 @@
+The logging configuration options are now registered with their types, so an invalid value fails with a usage error instead of being silently accepted: :confval:`log_file_mode` accepts only ``w`` and ``a``, :confval:`log_level`, :confval:`log_cli_level` and :confval:`log_file_level` an integer or a string, and :confval:`log_auto_indent` a boolean, an integer or a string. Their help text in ``pytest --help`` now describes each option instead of reading ``Default value for --log-cli-format``, and :confval:`log_cli` gained a ``--log-cli`` command line flag.

changelog/10551.improvement.2.rst

@@ -0,0 +1 @@
+:confval:`max_warnings` and :confval:`faulthandler_timeout` are now registered with their types. ``max_warnings`` is an integer, as it was already documented to be, and an invalid value is reported as a usage error at startup instead of crashing at the end of the run; ``faulthandler_timeout`` is a float.

changelog/10551.improvement.3.rst

@@ -0,0 +1,3 @@
+Declaring a configuration option that another plugin already declared, with a different type or default, now warns and says where both declarations are, instead of silently overwriting the first one.
+
+:func:`config.getini <pytest.Config.getini>` and :func:`config.getoption <pytest.Config.getoption>` also now say which of the two to use when the name is known to the other one.

changelog/14960.bugfix.rst

@@ -0,0 +1 @@
+A command line flag that a plugin registers as a shorthand for setting a configuration option now takes effect when the plugin is a third-party plugin or a ``conftest.py``. Previously only flags registered by pytest's own plugins were applied, and the others were silently ignored by :func:`config.getini <pytest.Config.getini>`.

changelog/9244.improvement.rst

@@ -0,0 +1 @@
+``pytest --help`` now shows the default value of each configuration option, and which option it falls back to if it has one. ``%(default)s`` in the help text passed to :func:`parser.addini <pytest.Parser.addini>` is substituted, as argparse already does for command line options.

doc/en/deprecations.rst

@@ -15,6 +15,37 @@ Below is a complete list of all pytest features which are considered deprecated.
 :class:`~pytest.PytestWarning` or subclasses, which can be filtered using :ref:`standard warning filters <warnings>`.
 
 
+.. _config-option-for-settings:
+
+``config.option`` for a configuration setting
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. deprecated:: 9.2
+
+A setting declared with :func:`parser.addconfig <pytest.Parser.addconfig>`, or
+with a command line option that overrides a configuration option, is resolved
+from the configuration files and the command line together. Reading it as
+``config.option.<name>`` is deprecated, and writing it there no longer changes
+the setting:
+
+.. code-block:: python
+
+    # Deprecated:
+    level = config.option.log_level
+    config.option.log_level = "DEBUG"
+
+    # Use instead:
+    level = config.getini("log_level")
+
+Reading it there used to be a trap: the attribute was ``None`` whenever the
+command line option was absent, even for a setting the configuration file set.
+It now holds the resolved value.
+
+Plain :func:`parser.addoption <pytest.Parser.addoption>` options are not
+configuration settings and are unaffected; ``config.option`` remains the way
+to read and write them.
+
+
 .. _callspec2-renamed:
 
 ``_pytest.python.CallSpec2`` renamed to ``CallSpec``

doc/en/how-to/writing_plugins.rst

@@ -309,6 +309,149 @@ If you want to look at the names of existing plugins, use
 the :option:`--trace-config` option.
 
 
+.. _declaring-config-options:
+
+Declaring configuration options
+-------------------------------
+
+.. versionadded:: 9.2
+
+.. note::
+
+    :func:`parser.addconfig() <pytest.Parser.addconfig>` and the ``fallback``
+    argument to :func:`parser.addini() <pytest.Parser.addini>` are
+    experimental. Their behaviour and signatures may change in future
+    releases.
+
+A setting that your plugin reads -- a name to greet, a format to print in, a
+limit to enforce -- is declared once, from the :hook:`pytest_addoption` hook,
+with :func:`parser.addconfig() <pytest.Parser.addconfig>`:
+
+.. code-block:: python
+
+    def pytest_addoption(parser):
+        parser.addconfig(
+            "hello_name",
+            'Name to greet, as in "Hello World!"',
+            type=str,
+            default="World",
+            cli="--hello-name",
+            metavar="NAME",
+            group="helloworld",
+        )
+
+This gives users both a ``hello_name`` configuration option and a
+``--hello-name`` command line option. There is one way to read it, whichever
+way the user set it:
+
+.. code-block:: python
+
+    @pytest.fixture
+    def hello(request):
+        def _hello(name=None):
+            return f"Hello {name or request.config.getini('hello_name')}!"
+
+        return _hello
+
+The command line option *overrides* the configuration option rather than being
+a separate value, which is what lets there be a single read. Leave ``cli`` out
+and the setting is configuration-only; users can still change it for one run
+with ``-o hello_name=Bob``, so a setting does not need a command line option
+merely to be overridable.
+
+Types
+~~~~~
+
+``type`` accepts ``str``, ``bool``, ``int`` and ``float``, a union of those
+such as ``int | str``, a ``Literal`` of strings for a fixed set of choices, and
+the list-valued tags ``"args"``, ``"linelist"`` and ``"paths"``:
+
+.. code-block:: python
+
+    from typing import Literal
+
+    parser.addconfig(
+        "hello_style",
+        "How enthusiastic to be",
+        type=Literal["plain", "shouting"],
+        default="plain",
+        cli="--hello-style",
+    )
+
+Declaring the type once is what makes ``-o hello_style=whispering`` a clean
+usage error: it is enforced wherever the value comes from -- a configuration
+file, ``-o``, or the command line option.
+
+A ``bool`` setting becomes a flag taking no argument. Pass ``cli_value`` to
+make a flag out of any other type, so that a ``--shout`` flag can set
+``hello_style`` to ``shouting``.
+
+Falling back to another setting
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When one setting refines another -- a format for one output stream, defaulting
+to the format used for all of them -- say so with ``fallback`` rather than
+resolving it at each place you read the value:
+
+.. code-block:: python
+
+    parser.addconfig("hello_name", "Name to greet", type=str, default="World")
+    parser.addconfig(
+        "hello_shout_name",
+        "Name to greet when shouting",
+        type=str,
+        default="World",
+        fallback="hello_name",
+    )
+
+``config.getini("hello_shout_name")`` then returns ``hello_name`` whenever
+``hello_shout_name`` itself is not configured. A fallback must be registered
+before the settings naming it, and must have the same type; several may be
+given, and the first one that is configured wins.
+
+Declaring the fallback also gets two things right that are easy to miss when
+writing the chain by hand: a setting explicitly configured to an empty value is
+honoured rather than falling through, and ``pytest --help`` says what this
+setting falls back to.
+
+Reading a setting
+~~~~~~~~~~~~~~~~~
+
+:func:`config.getini() <pytest.Config.getini>` reads a setting whichever way
+the user set it. :class:`config.settings <pytest.Settings>` is the same values
+as a mapping, and can also say where one came from:
+
+.. code-block:: python
+
+    config.settings["hello_name"]  # same as config.getini("hello_name")
+    config.settings.source_of("hello_name")  # Source.FILE, Source.CLI, ...
+    config.settings.spec("hello_name").default  # how it was declared
+
+Note that a setting is *not* read as ``config.option.<name>``. That still
+works, and shows the resolved value, but it is deprecated: writing it there
+does not change the setting. ``config.option`` remains the way to read the
+command line options that are not settings, which are also listed as
+``config.settings.options``.
+
+The underlying calls
+~~~~~~~~~~~~~~~~~~~~
+
+``addconfig`` is a convenience over two lower-level calls, which remain
+available and are what you want when a setting is not really a setting:
+
+* :func:`parser.addoption() <pytest.Parser.addoption>` registers a command line
+  option only, taking argparse's arguments and read with
+  :func:`config.getoption() <pytest.Config.getoption>`. Use it for things that
+  are not configuration at all -- selecting what to run, entering a debugger,
+  anything meaningless to write down in a file.
+* :func:`parser.addini() <pytest.Parser.addini>` registers a configuration
+  option only, read with :func:`config.getini() <pytest.Config.getini>`. It
+  also accepts ``fallback``.
+
+Note that their ``type`` arguments are unrelated: ``addoption`` takes an
+argparse converter, ``addini`` takes the types described above.
+
+
 .. _registering-markers:
 
 Registering custom markers
@@ -362,23 +505,22 @@ string value of ``Hello World!`` if we do not supply a value or ``Hello
 
 
     def pytest_addoption(parser):
-        group = parser.getgroup("helloworld")
-        group.addoption(
-            "--name",
-            action="store",
-            dest="name",
+        parser.addconfig(
+            "hello_name",
+            'Default "name" for hello().',
+            type=str,
             default="World",
-            help='Default "name" for hello().',
+            cli="--hello-name",
+            metavar="NAME",
+            group="helloworld",
         )
 
 
     @pytest.fixture
     def hello(request):
-        name = request.config.getoption("name")
-
         def _hello(name=None):
             if not name:
-                name = request.config.getoption("name")
+                name = request.config.getini("hello_name")
             return f"Hello {name}!"
 
         return _hello

doc/en/reference/reference.rst

@@ -977,6 +977,19 @@ Config
 .. autoclass:: pytest.Config()
     :members:
 
+Settings
+~~~~~~~~
+
+.. autoclass:: pytest.Settings()
+    :members:
+    :special-members: __getitem__, __iter__, __contains__
+
+.. autoclass:: pytest.Setting()
+    :members:
+
+.. autoclass:: pytest.Source()
+    :members:
+
 Dir
 ~~~
 
@@ -1314,6 +1327,9 @@ Custom warnings generated in some situations such as improper usage or deprecate
 .. autoclass:: pytest.PytestRemovedIn10Warning
   :show-inheritance:
 
+.. autoclass:: pytest.PytestRemovedIn11Warning
+  :show-inheritance:
+
 .. autoclass:: pytest.PytestUnknownMarkWarning
    :show-inheritance: