Using `zope.meta`¶

Purpose¶

Bring the configuration of the zopefoundation packages into a common state and keep it there.

Configuration types¶

This directory contains the configuration directories for different types of packages:

buildout-recipe
- Configuration for a zc.buildout recipe. It is tested using multiple processes, so coverage has to be configured in a special way. (Supports pure Python packages which also run on PyPy.)
c-code
- Configuration for package containing C code besides the Python one.
pure-python
- Configuration for a pure Python package.
zope-product
- Configuration for a pure Python package which uses zc.buildout inside tox.ini to be able to pin the installed dependency versions the same way buildout.cfg does it.
toolkit
- Configuration used for the zopetoolkit and groktoolkit repositories.

Configuration templates¶

Each configuration type folder can override the default configuration in the folder default by providing one or more of these files:

packages.txt
- This file lists the packages which use the configuration in the directory.
CONTRIBUTING.md
- This file is copied as is. It allows developers to easily find our contributing guidelines in the root of the repository.
editorconfig
- This file is copied to .editorconfig and allows developers to have a common editor configuration experience in all repos.
gitignore.j2
- This file is copied to .gitignore and can be appended trough configuration in .meta.cfg.
MANIFEST.in.j2
- Configuration file for the MANIFEST to include all needed files in sdist and wheel.
readthedocs.yaml.j2
- Configuration for https://readthedocs.org to build the documentation over there if the package has documentation.
setup.cfg.j2
- common setup.cfg, which should be copied to the repository of the package
tox.ini.j2
- tox configuration, which should be copied to the repository of the package
tests.yml.j2
- Configuration for GitHub actions.

The `config-package` script¶

The config-package script applies package configuration in a given Python package.

Preparation¶

The scripts needs a venv with some packages installed:

$ python3.11 -m venv .
$ bin/pip install .

To use the configuration provided here in a package call the following script:

$ bin/config-package <path-to-package> --type <config-type-name> [<additional-options>]

See --help for details.

The script does the following steps:

Add the package name to packages.txt of the selected configuration type if it is not yet added.
Copy setup.cfg, tox.ini, tests.yml, MANIFEST.in, .readthedocs.yaml (if needed), and .gitignore to the repository.
Create or update a pyproject.toml project configuration file.
Remove a possibly existing .coveragerc and bootstrap.py. (Coverage is now configured in tox.ini for packages which are no buildout recipes.)
Run the tests via: tox. The tox script may be either on the current $PATH or in the bin subfolder of the current working directory.
Create a branch and a pull request. (Prevent an automatic commit of all changes with the command line switch --no-commit, or an automatic push to GitHub using the command line switch --no-push.)

After running the script you should manually do the following steps:

Check for changes in the updated repository and for the need of a change log entry over there.
Make sure the package is activated on https://coveralls.io by trying to add the repository name and making it active.
Check in possible changes in the zopefoundation/meta repository.

CLI arguments¶

The following arguments are supported.

--commit-msg=MSG: Use MSG as commit message instead of an artificial one.
--no-commit: Don’t automatically commit changes after the configuration run. Implies –no-push.
--no-push: Avoid pushing at the end of the configuration run.
--no-tests: Don’t run the package’s unit tests as part of the configuration run. Useful for quickly testing iterative configuration changes.
--branch: Define a specific git branch name to be created for the changes. By default the script creates one which includes the name of the configuration type.

The following options are only needed one time as their values are stored in .meta.toml..

--type: Define the configuration type (see Configuration types section above) to be used for the repository.
--template-overrides: Filesystem path to a folder that contains subfolders for configuration type and default templates. Used to override built-in configuration templates. Empty override template files will prevent creating the respective file in the repository you are managing with zope.meta, this way you can purposely omit creating some optional files.
--with-macos: Enable running the tests on macOS on GitHub Actions.
--with-windows: Enable running the tests on Windows on GitHub Actions.
--with-pypy: Enable PyPy support.
--with-future-python: The package supports the next upcoming Python version which does not yet have a final release thus it is not yet generally supported by the zopefoundation packages.
--oldest-python: The oldest version of Python supported by this package. Specified as version number, e.g. 3.12. This setting is optional and defaults to the lowest Python version generally supported by zopefoundation packages.
--with-docs: Enable building the documentation using Sphinx. This will also create a configuration file .readthedocs.yaml for integration with https://readthedocs.org.
--with-sphinx-doctests: Enable running the documentation as doctest using Sphinx.

Options¶

It is possible to configure some options in a .meta.toml file inside the package repository. This file also stores the template name and commit id of the meta repository at the time of the run. This file is generated during the configuration run, if it does not exist or at least gets updated. Example:

[meta]
template = "pure-python"
commit-id = "< commit-hash >"

[python]
with-pypy = false
with-docs = true
with-sphinx-doctests = false
with-macos = false
with-windows = false

[coverage]
fail-under = 98

[coverage-run]
additional-config = [
    "data_file = $COVERAGE_HOME.coverage",
]
omit = [
    "src/foo/bar.py",
]
source = "src"

[tox]
additional-envlist = [
    "py311-slim",
    "py312-fat",
    ]
testenv-additional-extras = [
    "extra-feature",
    ]
testenv-commands-pre = [
    "{envbindir}/buildout -c ...",
    ]
testenv-commands = [
    "{envbindir}/test {posargs:-cv}",
    "{envbindir}/test_with_gs {posargs:-cv}",
    ]
testenv-deps = [
    "zope.testrunner",
    ]
testenv-setenv = [
    "ZOPE_INTERFACE_STRICT_IRO=1",
]
testenv-additional = [
    "passenv =",
    "    DISPLAY",
    ]
coverage-basepython = "python3.13"
coverage-command = [
    "coverage run {envbindir}/test_with_gs []",
    ]
coverage-setenv = [
    "COVERAGE_HOME={toxinidir}",
    ]
coverage-additional = [
    "depends = py312,docs",
    ]
docs-deps = [
    "urllib3 < 2",
    ]

[flake8]
additional-config = [
    "# D203 1 blank line required before class docstring",
    "# E221 multiple spaces before operator",
    "# E222 multiple spaces after operator",
    "# W503 Line break occurred before a binary operator",
    "per-file-ignores =",
    "    src/foo/bar.py: E221 E222",
    "extend-ignore = D203, W503",
    ]
additional-sources = "testproj foo bar.py"

[manifest]
additional-rules = [
    "include *.foo",
    "include *.bar",
    ]

[check-manifest]
additional-ignores = [
    "docs/html/*",
    "docs/source/_static/*",
    ]
ignore-bad-ideas = [
    "src/foo/bar.mo",
    ]

[isort]
known_third_party = "ipaddress, PasteDeploy"
known_zope = "AccessControl, Acquisition, App"
known_first_party = "Products.GenericSetup, Products.CMFCore"
additional-sources = "{toxinidir}/tests {toxinidir}/bar.py"

[github-actions]
services = [
    "postgres:",
    "  image: postgres",
    ]
additional-config = [
    "- [\"3.12\",   \"py312-slim\"]",
    ]
additional-exclude = [
    "- { os: windows, config: [\"pypy-3.10\", \"pypy3\"] }",
    "- { os: macos, config: [\"pypy-3.10\", \"pypy3\"] }",
    ]
steps-before-checkout = [
    "- name: \"Set some Postgres settings\"",
    "  run: ...",
    ]
additional-install = [
    "sudo apt-get update && sudo apt-get install -y libxml2-dev libxslt-dev",
    "pip install tox-factor"
    ]
additional-build-dependencies = [
    "cffi",
    "python-ldap",
    ]
test-enviroment = [
    "TEST_DSN: 'host=localhost port=5432 user=postgres'"
    ]
test-commands = [
    "tox -f ${{ matrix.config[1] }}",
    ]

[c-code]
manylinux-install-setup = [
    "export CFLAGS=\"-pipe\"",
    ]
manylinux-aarch64-tests = [
    "cd /io/",
    "\"${PYBIN}/pip\" install tox",
    "\"${PYBIN}/tox\" -e py",
    "cd ..",
    ]

[zest-releaser]
options = [
    "prereleaser.before =",
    "    zest.pocompile.compile.main",
    ]

[git]
ignore = [
    "*.mo",
    ]

[pre-commit]
teyit-exclude = "App/tests/fixtures/error\.py"
pyupgrade-exclude = "^src/zope/proxy/__init__\.py$"


[readthedocs]
build-extra = [
    "apt_packages:",
    "  - libldap2-dev",
    ]

Meta Options¶

template: Name of the configuration type, to be used as the template for the repository. Currently read-only.
commit-id: Commit of the meta repository, which was used for the last configuration run. Currently read-only.

Python options¶

with-macos: Run the tests also on macOS on GitHub Actions: true/false, default: false
with-windows: Run the tests also on Windows on GitHub Actions: true/false, default: false
with-pypy: Does the package support PyPy: true/false
with-docs: Build the documentation via Sphinx: true/false
with-sphinx-doctests: Run the documentation as doctest using Sphinx: true/false

Coverage options¶

The corresponding section is named: [coverage].

fail-under: A minimal value of code coverage below which a test failure is issued.

Coverage:run options¶

The corresponding section is named: [coverage-run].

additional-config: Additional options for the [run] section of the coverage configuration. This option has to be a list of strings.
omit: Files to be omitted from the coverage report. This option has to be a list of strings. It defaults to an empty list.
source: This option defines the value of source in the coverage [run] section. This option has to be a string. It defaults to the name of the package if it is not set.

tox.ini options¶

The corresponding section is named: [tox].

additional-envlist: This option contains additional entries for the envlist in tox.ini. The configuration for the needed additional environments can be added using testenv-additional (see below). This option has to be a list of strings without indentation.
testenv-additional-extras: Additional entries for the extras option in [testenv] of tox.ini. This option has to be a list of strings without indentation.
testenv-commands-pre: Replacement for the default commands_pre option in [testenv] of tox.ini. This option has to be a list of strings without indentation.
testenv-commands: Replacement for the default commands option in [testenv] of tox.ini. This option has to be a list of strings without indentation.
testenv-deps: Additional dependencies for the deps option in [testenv] of tox.ini. This option has to be a list of strings without indentation. It is empty by default.
testenv-setenv: Set the value of the setenv option in [testenv] of tox.ini. Depending in the template used this might be an addition to the predefined values for this option. This option has to be a list of strings.
testenv-additional: Additional lines for the section [testenv] in tox.ini. This option has to be a list of strings.
coverage-basepython: This option replaces the value for the basepython option in the section [testenv:coverage]. This option has to be a string. The default value is python3.
coverage-command: This option replaces the coverage call in the section [testenv:coverage] in tox.ini. Caution: only the actual call to collect the coverage data is replaced. The calls to create the reporting are not changed. This option has to be a list or a string. If it is not set or empty the default is used.
coverage-setenv: This option defines the contents for the option setenv in the section [testenv:coverage] in tox.ini. If it has a default value (e. g. as in the buildout-recipe template), the default value is replaced by the value given here. This option has to be a list of strings.
coverage-additional: This option allows to add additional lines below [testenv:coverage] in tox.ini. This option has to be a list of strings.
docs-deps: This option allows to add additional install dependencies for [testenv:docs] in tox.ini. This option has to be a list of strings and is empty by default. Caution: The values set for this option override the ones set in [testenv].

Flake8 options¶

The corresponding section is named: [flake8].

additional-config: Additional configuration options be added at the end of the flake8 configuration section in setup.cfg. Caution: This option has to be a list of strings so the leading white spaces and comments are preserved when writing the value to setup.cfg.
additional-sources: Sometimes not only src and setup.py contain Python code to be checked by flake8. Additional files or directories can be configured here. This option is a string. The sources inside have to be space separated.

Manifest options¶

The corresponding section is named: [manifest].

additional-rules: Additional rules to be added at the end of the MANIFEST.in file. This option has to be a list of strings.

Check-manifest options¶

The corresponding section is named: [check-manifest].

additional-ignores: Additional files to be ignored by check-manifest via its section in setup.cfg. This option has to be a list of strings.
ignore-bad-ideas: Ignore bad idea files/directories matching these patterns. This option has to be a list of strings.

Isort options¶

The corresponding section is named: [isort].

Please note the usage of underscores for the option name, which used to be consistent with the name of the option in isort.

Currently only the configuration type zope-product supports isort configurations.

known_third_party: This option defines the value for known_third_party in the isort configuration. This option has to be a string. It defaults to "docutils, pkg_resources, pytz".
known_zope: This option defines the value for known_zope in the isort configuration. This option has to be a string. It defaults to the empty string.
known_first_party: This option defines the value for known_first_party in the isort configuration. This option has to be a string. It defaults to the empty string.
known_local_folder: This option defines the value for known_local_folder in the isort configuration. This option has to be a string. It defaults to the empty string.
additional-sources: This option defines additional files and/or directories where isort should be applied. This option has to be a string. It defaults to the empty string.
additional-config: Additional options for the [isort] section. This option has to be a list of strings.

GitHub Actions options¶

The corresponding section is named: [github-actions].

services: Lines which will be added in the services section of the GitHub Actions build section. This option has to be a list of strings.
additional-config: Additional entries for the config matrix. This option has to be a list of strings without leading whitespace but it has to start with a hyphen.
additional-exclude: Additional entries to exclude from the config matrix. This option has to be a list of strings without leading whitespace but it has to start with a hyphen.
steps-before-checkout: Add steps definitions to be inserted into tests.yml before the checkout action i. e. as the first step. This option has to be a list of strings.
additional-install: Additional lines to be executed during the install dependencies step when running the tests on GitHub Actions. This option has to be a list of strings. For the template c-code this option is currently used to replace how to install the package itself and run tests and coverage.
test-environment: Environment variables to be set during the test run. This option has to be a list of strings.
test-commands: Replacement for the test command in tests.yml. This option has to be a list of strings.

C-code options¶

The corresponding section is named: [c-code] it is used only for packages built with the template c-code.

manylinux-install-setup: Additional setup steps necessary in manylinux-install.sh. This option has to be a list of strings and defaults to an empty list.
manylinux-aarch64-tests: Replacement for the tests against the aarch64 architecture. This option has to be a list of strings and defaults to testing using tox against all supported Python versions, which could be too slow for some packages.

zest.releaser options¶

The corresponding section is named: [zest-releaser] (with an - instead of the .).

options: (Additional) options used to configure zest.releaser. This option has to be a list of strings and defaults to an empty list.

git options¶

The corresponding section is named: [git].

ignore: Additional lines to be added to the .gitignore file. This option has to be a list of strings and defaults to an empty list.

pre-commit options¶

The corresponding section is named: [pre-commit].

teyit-exclude: Regex for files to be hidden from teyit. It fails on files containing syntax errors. This option has to be a string and is omitted when not defined.
pyupgrade-exclude: Regex for files to be hidden from pyupgrade. It might be a bit overly optimistic with its changes. This option has to be a string and is omitted when not defined.

ReadTheDocs options¶

The corresponding section is named: [readthedocs].

build-extra: Additional lines to be added to the build configuration in the ReadTheDocs configuration file .readthedocs.yaml. This option has to be a list of strings and defaults to an empty list.

Configuration script hints¶

Calling config-package again updates a previously created pull request if there are changes made in the files config-package touches.
Call bin/check-python-versions <path-to-package> -h to see how to fix version mismatches in the lint tox environment.

Other helper scripts¶

Updating to the currently supported Python versions¶

There is a script update-python-support for updating a repository to the currently supported Python versions as defined in shared/package.py.

Usage¶

To update a repository to the currently supported Python versions call:

$ bin/update-python-support <path-to-package>

It supports a parameter --interactive to gather user input for its changes and not automatically commit them. It also supports a parameter --no-commit that prevents automatic commits but attempts to cut down on interactively asking for user input. Some of that still happens due to limitations of the zest.releaser scripts used by update-python-support.

Calling a script on multiple repositories¶

The config-package script only runs on a single repository. To update multiple repositories at once you can use multi-call. It runs a given script on all repositories listed in a packages.txt file.

Usage¶

To run a script on all packages listed in a packages.txt file call multi-call the following way:

$ bin/multi-call <name-of-the-script> <path-to-packages.txt> <path-to-clones> <arguments-for-script>

See --help for details.

The script does the following steps:

It does the following steps for each line in the given packages.txt which does not start with #.
Check if there is a repository in <path-to-clones> with the name of the repository. If it does not exist: clone it. If it exists: clean the clone from changes, switch to master branch and pull from origin.
Call the given script with the package name and arguments for the script.

Caution

Running this script stashes any uncommitted changes in the repositories, run git stash pop to recover them.

Re-enabling GitHub Actions¶

After a certain period of time (currently 60 days) without commits GitHub automatically disables Actions. They can be re-enabled manually per repository. There is a script to do this for all repositories. It does no harm if Actions is already enabled for a repository.

Preparation¶

Install GitHub’s CLI application, see https://github.com/cli/cli.
Authorize using the application:
- gh auth login
- It is probably enough to do it once.

Usage¶

To run the script just call it:

$ bin/re-enable-actions

Using zope.meta¶

Purpose¶

Configuration types¶

Configuration templates¶

The config-package script¶

Preparation¶

CLI arguments¶

Options¶

Meta Options¶

Python options¶

Coverage options¶

Coverage:run options¶

tox.ini options¶

Flake8 options¶

Manifest options¶

Check-manifest options¶

Isort options¶

GitHub Actions options¶

C-code options¶

zest.releaser options¶

git options¶

pre-commit options¶

ReadTheDocs options¶

Configuration script hints¶

Other helper scripts¶

Updating to the currently supported Python versions¶

Usage¶

Calling a script on multiple repositories¶

Usage¶

Re-enabling GitHub Actions¶

Preparation¶

Usage¶

Using `zope.meta`¶

The `config-package` script¶