Using zope.meta

Purpose

Bring the configuration of the zopefoundation packages into a common state and keep it there.

Configuration types

This directory contains the configuration directories for different types of packages:

  • buildout-recipe

    • Configuration for a zc.buildout recipe. It is tested using multiple processes, so coverage has to be configured in a special way. (Supports pure Python packages which also run on PyPy.)

  • c-code

    • Configuration for package containing C code besides the Python one.

  • pure-python

    • Configuration for a pure Python package.

  • zope-product

    • Configuration for a pure Python package which uses zc.buildout inside tox.ini to be able to pin the installed dependency versions the same way buildout.cfg does it.

  • toolkit

    • Configuration used for the zopetoolkit and groktoolkit repositories.

Configuration templates

Each configuration type folder can override the default configuration in the folder default by providing one or more of these files:

  • packages.txt

    • This file lists the packages which use the configuration in the directory.

  • CONTRIBUTING.md

    • This file is copied as is. It allows developers to easily find our contributing guidelines in the root of the repository.

  • editorconfig

    • This file is copied to .editorconfig and allows developers to have a common editor configuration experience in all repos.

  • gitignore.j2

    • This file is copied to .gitignore and can be appended trough configuration in .meta.cfg.

  • MANIFEST.in.j2

    • Configuration file for the MANIFEST to include all needed files in sdist and wheel.

  • readthedocs.yaml.j2

    • Configuration for https://readthedocs.org to build the documentation over there if the package has documentation.

  • setup.cfg.j2

    • common setup.cfg, which should be copied to the repository of the package

  • tox.ini.j2

    • tox configuration, which should be copied to the repository of the package

  • tests.yml.j2

    • Configuration for GitHub actions.

The config-package script

The config-package script applies package configuration in a given Python package.

Preparation

The scripts needs a venv with some packages installed:

$ python3.11 -m venv .
$ bin/pip install .

To use the configuration provided here in a package call the following script:

$ bin/config-package <path-to-package> --type <config-type-name> [<additional-options>]

See --help for details.

The script does the following steps:

  1. Add the package name to packages.txt of the selected configuration type if it is not yet added.

  2. Copy setup.cfg, tox.ini, tests.yml, MANIFEST.in, .readthedocs.yaml (if needed), and .gitignore to the repository.

  3. Create or update a pyproject.toml project configuration file.

  4. Remove a possibly existing .coveragerc and bootstrap.py. (Coverage is now configured in tox.ini for packages which are no buildout recipes.)

  5. Run the tests via: tox. The tox script may be either on the current $PATH or in the bin subfolder of the current working directory.

  6. Create a branch and a pull request. (Prevent an automatic commit of all changes with the command line switch --no-commit, or an automatic push to GitHub using the command line switch --no-push.)

After running the script you should manually do the following steps:

  1. Check for changes in the updated repository and for the need of a change log entry over there.

  2. Make sure the package is activated on https://coveralls.io by trying to add the repository name and making it active.

  3. Check in possible changes in the zopefoundation/meta repository.

CLI arguments

The following arguments are supported.

--commit-msg=MSG

Use MSG as commit message instead of an artificial one.

--no-commit

Don’t automatically commit changes after the configuration run. Implies –no-push.

--no-push

Avoid pushing at the end of the configuration run.

--no-tests

Don’t run the package’s unit tests as part of the configuration run. Useful for quickly testing iterative configuration changes.

--branch

Define a specific git branch name to be created for the changes. By default the script creates one which includes the name of the configuration type.

The following options are only needed one time as their values are stored in .meta.toml..

--type

Define the configuration type (see Configuration types section above) to be used for the repository.

--overrides

Filesystem path to a folder containing customizations for applying zope.meta scripts. To override configuration templates, create a folder for the configuration type inside the overrides folder and add templates with the same name as the standard templates in zope.meta. Those will then be used instead of the default templates. Empty override template files will prevent creating the respective file in the repository you are managing with zope.meta, this way you can purposely omit creating some optional files. You can also override some configuration variables from the zope.meta.shared.packages module. Create a TOML file overrides.toml in the root of the overrides folder and add the desired values there. Useful variables to override include e.g. ORG for the organization name, the Python versions designated as OLDEST_PYTHON_VERSION and NEWEST_PYTHON_VERSION, or the META_HINT variables that point to the template sources.

--with-macos

Enable running the tests on macOS on GitHub Actions.

--with-windows

Enable running the tests on Windows on GitHub Actions.

--with-pypy

Enable PyPy support.

--with-future-python

The package supports the next upcoming Python version which does not yet have a final release thus it is not yet generally supported by the zopefoundation packages.

--with-free-threaded-python

Enable testing with the free-threaded (nogil) build of the newest supported Python version. Tests run on Linux only. For c-code packages, both the C extension and pure-Python variants are tested.

--oldest-python

The oldest version of Python supported by this package. Specified as version number, e.g. 3.12. This setting is optional and defaults to the lowest Python version generally supported by zopefoundation packages.

--with-docs

Enable building the documentation using Sphinx. This will also create a configuration file .readthedocs.yaml for integration with https://readthedocs.org.

--with-sphinx-doctests

Enable running the documentation as doctest using Sphinx.

Options

It is possible to configure some options in a .meta.toml file inside the package repository. This file also stores the template name and commit id of the meta repository at the time of the run. This file is generated during the configuration run, if it does not exist or at least gets updated. Example:

[meta]
template = "pure-python"
commit-id = "< commit-hash >"

[python]
with-pypy = false
with-docs = true
with-sphinx-doctests = false
with-macos = false
with-windows = false
with-free-threaded-python = false

[coverage]
fail-under = 98

[coverage-run]
additional-config = [
    "data_file = $COVERAGE_HOME.coverage",
]
omit = [
    "src/foo/bar.py",
]
source = "src"

[tox]
additional-envlist = [
    "py311-slim",
    "py312-fat",
    ]
testenv-additional-extras = [
    "extra-feature",
    ]
testenv-commands-pre = [
    "{envbindir}/buildout -c ...",
    ]
testenv-commands = [
    "{envbindir}/test {posargs:-cv}",
    "{envbindir}/test_with_gs {posargs:-cv}",
    ]
testenv-deps = [
    "zope.testrunner",
    ]
testenv-setenv = [
    "ZOPE_INTERFACE_STRICT_IRO=1",
]
testenv-additional = [
    "passenv =",
    "    DISPLAY",
    ]
coverage-basepython = "python3.13"
coverage-command = [
    "coverage run {envbindir}/test_with_gs []",
    ]
coverage-setenv = [
    "COVERAGE_HOME={toxinidir}",
    ]
coverage-additional = [
    "depends = py312,docs",
    ]
docs-deps = [
    "urllib3 < 2",
    ]
lint-diff-on-failure = true

[flake8]
additional-config = [
    "# D203 1 blank line required before class docstring",
    "# E221 multiple spaces before operator",
    "# E222 multiple spaces after operator",
    "# W503 Line break occurred before a binary operator",
    "per-file-ignores =",
    "    src/foo/bar.py: E221 E222",
    "extend-ignore = D203, W503",
    ]
additional-sources = "testproj foo bar.py"

[manifest]
additional-rules = [
    "include *.foo",
    "include *.bar",
    ]

[check-manifest]
additional-ignores = [
    "docs/html/*",
    "docs/source/_static/*",
    ]
ignore-bad-ideas = [
    "src/foo/bar.mo",
    ]

[isort]
known_third_party = "ipaddress, PasteDeploy"
known_zope = "AccessControl, Acquisition, App"
known_first_party = "Products.GenericSetup, Products.CMFCore"
additional-sources = "{toxinidir}/tests {toxinidir}/bar.py"

[github-actions]
services = [
    "postgres:",
    "  image: postgres",
    ]
additional-config = [
    "- [\"3.12\",   \"py312-slim\"]",
    ]
additional-exclude = [
    "- { os: windows, config: [\"pypy-3.10\", \"pypy3\"] }",
    "- { os: macos, config: [\"pypy-3.10\", \"pypy3\"] }",
    ]
steps-before-checkout = [
    "- name: \"Set some Postgres settings\"",
    "  run: ...",
    ]
additional-install = [
    "sudo apt-get update && sudo apt-get install -y libxml2-dev libxslt-dev",
    "pip install tox-factor"
    ]
additional-build-dependencies = [
    "cffi",
    "python-ldap",
    ]
test-enviroment = [
    "TEST_DSN: 'host=localhost port=5432 user=postgres'"
    ]
test-commands = [
    "tox -f ${{ matrix.config[1] }}",
    ]

[c-code]
manylinux-install-setup = [
    "export CFLAGS=\"-pipe\"",
    ]
manylinux-aarch64-tests = [
    "cd /io/",
    "\"${PYBIN}/pip\" install tox",
    "\"${PYBIN}/tox\" -e py",
    "cd ..",
    ]

[pypi]
trusted-publishing = true

[zest-releaser]
options = [
    "prereleaser.before =",
    "    zest.pocompile.compile.main",
    ]

[git]
ignore = [
    "*.mo",
    ]

[pre-commit]
teyit-exclude = "App/tests/fixtures/error\.py"
pyupgrade-exclude = "^src/zope/proxy/__init__\.py$"


[readthedocs]
build-extra = [
    "apt_packages:",
    "  - libldap2-dev",
    ]

Meta Options

template

Name of the configuration type, to be used as the template for the repository. Currently read-only.

commit-id

Commit of the meta repository, which was used for the last configuration run. Currently read-only.

Python options

with-macos

Run the tests also on macOS on GitHub Actions: true/false, default: false

with-windows

Run the tests also on Windows on GitHub Actions: true/false, default: false

with-pypy

Does the package support PyPy: true/false

with-docs

Build the documentation via Sphinx: true/false

with-sphinx-doctests

Run the documentation as doctest using Sphinx: true/false

with-free-threaded-python

Run the tests also with free-threaded (nogil) Python on Linux: true/false, default: false

Coverage options

The corresponding section is named: [coverage].

fail-under

A minimal value of code coverage below which a test failure is issued.

Coverage:run options

The corresponding section is named: [coverage-run].

additional-config

Additional options for the [run] section of the coverage configuration. This option has to be a list of strings.

omit

Files to be omitted from the coverage report. This option has to be a list of strings. It defaults to an empty list.

source

This option defines the value of source in the coverage [run] section. This option has to be a string. It defaults to the name of the package if it is not set.

tox.ini options

The corresponding section is named: [tox].

additional-envlist

This option contains additional entries for the envlist in tox.ini. The configuration for the needed additional environments can be added using testenv-additional (see below). This option has to be a list of strings without indentation.

testenv-additional-extras

Additional entries for the extras option in [testenv] of tox.ini. This option has to be a list of strings without indentation.

testenv-skip-test-extra

Don’t add the standard test in the extras option in [testenv] of tox.ini. This option has to be true/false. Default is false.

testenv-commands-pre

Replacement for the default commands_pre option in [testenv] of tox.ini. This option has to be a list of strings without indentation.

testenv-commands

Replacement for the default commands option in [testenv] of tox.ini. This option has to be a list of strings without indentation.

testenv-deps

Additional dependencies for the deps option in [testenv] of tox.ini. This option has to be a list of strings without indentation. It is empty by default.

testenv-setenv

Set the value of the setenv option in [testenv] of tox.ini. Depending in the template used this might be an addition to the predefined values for this option. This option has to be a list of strings.

testenv-additional

Additional lines for the section [testenv] in tox.ini. This option has to be a list of strings.

coverage-basepython

This option replaces the value for the basepython option in the section [testenv:coverage]. This option has to be a string. The default value is python3.

coverage-command

This option replaces the coverage call in the section [testenv:coverage] in tox.ini. Caution: only the actual call to collect the coverage data is replaced. The calls to create the reporting are not changed. This option has to be a list or a string. If it is not set or empty the default is used.

coverage-setenv

This option defines the contents for the option setenv in the section [testenv:coverage] in tox.ini. If it has a default value (e. g. as in the buildout-recipe template), the default value is replaced by the value given here. This option has to be a list of strings.

coverage-additional

This option allows to add additional lines below [testenv:coverage] in tox.ini. This option has to be a list of strings.

docs-deps

This option allows to add additional install dependencies for [testenv:docs] in tox.ini. This option has to be a list of strings and is empty by default. Caution: The values set for this option override the ones set in [testenv].

lint-diff-on-failure

In the past, the lint step always called pre-commit with the option --show-diff-on-failure, which meant any linting failures would automatically dump a diff with any outstanding changes in the entire package, even if they are unrelated to linting, to the console. This is not helpful unless there are very few changes and the diff is manageable. Setting this option to false prevents showing the diff. If not set, the default is true for backwards compatibility.

Flake8 options

The corresponding section is named: [flake8].

additional-config

Additional configuration options be added at the end of the flake8 configuration section in setup.cfg. Caution: This option has to be a list of strings so the leading white spaces and comments are preserved when writing the value to setup.cfg.

additional-sources

Sometimes not only src and setup.py contain Python code to be checked by flake8. Additional files or directories can be configured here. This option is a string. The sources inside have to be space separated.

Manifest options

The corresponding section is named: [manifest].

additional-rules

Additional rules to be added at the end of the MANIFEST.in file. This option has to be a list of strings.

Check-manifest options

The corresponding section is named: [check-manifest].

additional-ignores

Additional files to be ignored by check-manifest via its section in setup.cfg. This option has to be a list of strings.

ignore-bad-ideas

Ignore bad idea files/directories matching these patterns. This option has to be a list of strings.

Isort options

The corresponding section is named: [isort].

Please note the usage of underscores for the option name, which used to be consistent with the name of the option in isort.

Currently only the configuration type zope-product supports isort configurations.

known_third_party

This option defines the value for known_third_party in the isort configuration. This option has to be a string. It defaults to "docutils, pkg_resources, pytz".

known_zope

This option defines the value for known_zope in the isort configuration. This option has to be a string. It defaults to the empty string.

known_first_party

This option defines the value for known_first_party in the isort configuration. This option has to be a string. It defaults to the empty string.

known_local_folder

This option defines the value for known_local_folder in the isort configuration. This option has to be a string. It defaults to the empty string.

additional-sources

This option defines additional files and/or directories where isort should be applied. This option has to be a string. It defaults to the empty string.

additional-config

Additional options for the [isort] section. This option has to be a list of strings.

GitHub Actions options

The corresponding section is named: [github-actions].

services

Lines which will be added in the services section of the GitHub Actions build section. This option has to be a list of strings.

additional-config

Additional entries for the config matrix. This option has to be a list of strings without leading whitespace but it has to start with a hyphen.

additional-exclude

Additional entries to exclude from the config matrix. This option has to be a list of strings without leading whitespace but it has to start with a hyphen.

steps-before-checkout

Add steps definitions to be inserted into tests.yml before the checkout action i. e. as the first step. This option has to be a list of strings.

additional-install

Additional lines to be executed during the install dependencies step when running the tests on GitHub Actions. This option has to be a list of strings. For the template c-code this option is currently used to replace how to install the package itself and run tests and coverage.

test-environment

Environment variables to be set during the test run. This option has to be a list of strings.

test-commands

Replacement for the test command in tests.yml. This option has to be a list of strings.

C-code options

The corresponding section is named: [c-code] it is used only for packages built with the template c-code.

manylinux-install-setup

Additional setup steps necessary in manylinux-install.sh. This option has to be a list of strings and defaults to an empty list.

manylinux-aarch64-tests

Replacement for the tests against the aarch64 architecture. This option has to be a list of strings and defaults to testing using tox against all supported Python versions, which could be too slow for some packages.

PyPI options

The corresponding section is named: [pypi] and its options configure the package publishing process to the PyPI Python package repository at https://pypi.org/.

trusted-publishing

Explicitly enable or disable automated building and publishing of release packages to PyPI using the Trusted Publishing process when a tag is pushed to GitHub. Defaults to false. This option has no meaning for c-code template projects, those always build and publish release packages using Trusted Publishing. See https://docs.pypi.org/trusted-publishers/ for details on the Trusted Publishing process and Other topics for how to configure a package for Trusted Publishing.

zest.releaser options

The corresponding section is named: [zest-releaser] (with an - instead of the .).

options

(Additional) options used to configure zest.releaser via the [tool.zest-releaser] section in pyproject.toml. This option has to be a list of strings and defaults to an empty list.

git options

The corresponding section is named: [git].

ignore

Additional lines to be added to the .gitignore file. This option has to be a list of strings and defaults to an empty list.

pre-commit options

The corresponding section is named: [pre-commit].

teyit-exclude

Regex for files to be hidden from teyit. It fails on files containing syntax errors. This option has to be a string and is omitted when not defined.

pyupgrade-exclude

Regex for files to be hidden from pyupgrade. It might be a bit overly optimistic with its changes. This option has to be a string and is omitted when not defined.

ReadTheDocs options

The corresponding section is named: [readthedocs].

build-extra

Additional lines to be added to the build configuration in the ReadTheDocs configuration file .readthedocs.yaml. This option has to be a list of strings and defaults to an empty list.

Configuration script hints

  • Calling config-package again updates a previously created pull request if there are changes made in the files config-package touches.

  • Call bin/check-python-versions <path-to-package> -h to see how to fix version mismatches in the lint tox environment.