Multi-Python, Single-Spec Macro System
This repository contains a work-in-progress macro system generator for the singlespec Python initiative. The macro system can be used in spec files for building RPM packages.
The purpose of the singlespec system is to take a package for a particular flavor, and autogenerate subpackages for all the other flavors.
Terminology
flavor is a kind of python interpreter. At this point, we recognize the following flavors:
python2
, python3
and pypy3
.
For compatibility reasons you see sometimes python
. In most places,
using python
is either a redefinition of python2
, or an alternative for
"flavor-agnostic". Conditionals are in place to switch python
to mean python3
in the future.
The name of the flavor is the name of the binary in /usr/bin
. It is also used as a prefix
for all flavor-specific macros. Some macros are redefined with "short" flavor for compatibility
reasons, such as py2
for python2
. All of them have a "long" form too.
modname is the PyPI name, or, if the package in question is not on PyPI, the moniker that we chose to stand in for it.
Packages adhering to the SUSE Python module naming policy are usually called %{flavor}-%{modname}
.
In some cases, it is only %{modname}
though.
pkgname, or subpackage name, is internal to a spec file, and is that thing you put after the
%package
macro. Pkgname of the package itself is an empty string. Pkgname of a %package -n something
is at this point -n something
, and denotes that this subpackage should not be handled
by the generator.
That means, if you want a subpackage to be skipped, rename it from %package foo
to
%package -n %{name}-foo
.
The purpose of the singlespec system is to take a package called %{flavor}-%{modname}
for a
particular flavor, and autogenerate subpackages for all the other flavors.
Alternately, it is to take package python-%{modname}
and generate subpackages for all flavors,
leaving the top-level package empty.
Build Set
The default build set is listed in the %pythons
macro. Every entry in %pythons
generates a
requirement in %python_module
, a subpackage from %python_subpackages
(unless the top-level spec
file is for that flavor), and an additional run of loops like %python_build
, _install
, _exec
and _expand
.
To control the build set, you can either completely redefine %pythons
, or exclude
particular flavor(s) by defining %skip_$flavor
. For example, if you %define skip_python2 1
,
then Python 2 will be excluded from the default build set.
Skip-macros are intended for per-package use only. Never define a skip-macro in prjconf or
in any other sort of global config. Instead, redefine %pythons
.
Macros
The following macros are considered public API:
-
%system_python
- flavor that is used for generic unflavored%python_
macros. Currently set topython2
. -
%python_for_executables
- flavor that is used for installing executables into%_bindir
and other files in non-flavor-specific locations. By default, set topython3
. -
%pythons
- the build set. See above for details. -
%have_python2
,%have_python3
,%have_pypy3
. Defined as 1 if the flavor is present in the build environment. Undefined otherwise.
Note: "present in build environment" does not mean "part of build set". Under some circumstances, you can get a Python flavor pulled in through dependencies, even if you exclude it from the build set. In such case,%have_$flavor
will be defined but packages will not be generated for it. -
%skip_python2
,%skip_python3
,%skip_pypy3
. Undefined by default. Define in order to exclude a flavor from build set. -
%{python_module modname [= version]}
expands to$flavor-modname [= version]
for every flavor. Intended as:BuildRequires: %{python_module foo}
. -
%{python_dist_name modname}
. Given a standardized name (i.e. dist name, name on PyPI) ofmodname
, it will convert it to a canonical format. -
%{python2_dist modname}
. Given a standardized name (i.e. dist name, name on PyPI) ofmodname
, it will convert it to a canonical format, and evaluates to python2.Ydist(CANONICAL_NAME), which is useful when listing dependencies. Intended as(Build)Requires: %{python2_dist foo}
. -
%{python3_dist modname}
. Given a standardized name (i.e. dist name, name on PyPI) ofmodname
, it will convert it to a canonical format, and evaluates to python3.Ydist(CANONICAL_NAME), which is useful when listing dependencies. Intended as(Build)Requires: %{python3_dist foo}
. -
%python_flavor
expands to the%pythons
entry that is currently being processed.
Does not apply in%prep
,%build
,%install
and%check
sections, except when evaluated as%{$python_flavor}
in%python_expand
. -
%ifpython2
,%ifpython3
,%ifpypy3
: applies the following section only to subpackages of that particular flavor.
%ifpycache
: applies the following section only to subpackages of flavors that generate a__pycache__
directory.
Note: These are shortcuts for%if "%python_flavor" == "$flavor"
. Due to how RPM evaluates the shortcuts, they will fail when nested with other%if
conditions. If you need to nest your conditions, use the full%if %python_flavor
spelling. -
%python2_only
,%python3_only
,%pypy3_only
: applies the contents of the line only to subpackages of that particular flavor. -
%pycache_only
: applies the contents of the line only to subpackages of flavors that generate__pycache__
directories. Useful in filelists:%pycache_only %{python_sitelib}/__pycache__/*
-
%python_build
expands to build instructions for all flavors. -
%python_install
expands to install instructions for all flavors. -
%python_exec something.py
expands to$flavor something.py
for all flavors, and moves around the distutils-generatedbuild
directory so that you are never runningpython2
script with a python3-generatedbuild
. This is only useful for distutils/setuptools. -
%python_expand something
is a more general form of the above. Performs the moving-around for distutils'build
directory, and performs rpm macro expansion of its argument for every flavor.
Importantly,$python
is replaced by current flavor name, even in macros. So:
%{python_expand $python generatefile.py %$python_bin_suffix}
expands to:
python2 generatefile.py %python2_bin_suffix
python3 generatefile.py %python3_bin_suffix
etc. -
%pytest
runspytest
in all flavors with appropriate environmental variables (namely, it sets$PYTHONPATH
to%{python_sitelib}
). All paramteres to this macro are passed without change to the pytest command. ExplicitBuildRequires
on%{python_module pytest}
is still required. -
%pytest_arch
the same as the above, except it sets$PYTHONPATH
to%{$python_sitearch}
-
%python_clone filename
creates a copy offilename
under a flavor-specific name for every flavor. This is useful for packages that install unversioned executables:/usr/bin/foo
is copied to/usr/bin/foo-%{python_bin_suffix}
for all flavors, and the shebang is modified accordingly.
%python_clone -a filename
will also invoke%prepare_alternative
with the appropriate arguments. -
%python2_build
,%python3_build
,%pypy3_build
expands to build instructions for the particular flavor. -
%python2_install
,%python3_install
,%pypy3_install
expands to install instructions for the particular flavor. -
%python_subpackages
expands to the autogenerated subpackages. This should go at the end of the main headers section. -
%python_enable_dependency_generator
expands to a define to enable automatic requires generation of Python module dependencies using egg-info/dist-info metadata. This should go above the%python_subpackages
macro, preferably closer to the top of the spec. Intended usage:%{?python_enable_dependency_generator}
. This macro will eventually be removed when the generator is configured to automatically run, hence the?
at the beginning of the macro invocation.
Alternative-related, general:
-
%prepare_alternative [-t <targetfile> ] <name>
replaces<targetfile>
with a symlink to/etc/alternatives/<name>
, plus related housekeeping. If no<targetfile>
is given, it is%{_bindir}/<name>
. -
%install_alternative [-n ]<name> [-s <sourcefile>] [-t ]<targetfile> [-p ]<priority>
runs theupdate-alternative
command, configuring<sourcefile>
alternative to be<targetfile>
, with a priority<priority>
. If no<sourcefile>
is given, it is%{_bindir}/<name>
. Can be followed by additional arguments toupdate-alternatives
, such as--slave
. -
%uninstall_alternative [-n ]<name> [-t ]<targetfile>
if uninstalling (not upgrading) the package, remove<targetfile>
from a list of alternatives under<name>
-
%alternative_to <file>
generates a filelist entry for<file>
and a ghost entry forbasename <file>
in/etc/alternatives
Alternative-related, for Python:
-
%python_alternative <file>
: expands to filelist entries for<file>
, its symlink in/etc/alternatives
, and the target file called<file>-%python_bin_suffix
.
In case the file is a manpage (file.1.gz
), the target is calledfile-%suffix.1.gz
. -
%python_install_alternative <name> [<name> <name>...]
: runsupdate-alternatives
for<name>-%{python_bin_suffix}
. If more than one argument is present, the remaining ones are converted to--slave
arguments. If aname
is in the form ofsomething.1
orsomething.4.gz
(any number applies), it is handled as a manpage and assumed to live in the appropriate%{_mandir}
subdirectory, otherwise it is handled as a binary and assumed to live in%{_bindir}
. You can also supply a full path to override this behavior. -
%python_uninstall_alternative <name>
: reverse of the preceding.
Note that if you created a group by specifying multiple arguments toinstall_alternative
, only the first one applies foruninstall_alternative
.Each of these has a flavor-specific spelling:
%python2_alternative
etc.
In addition, the following flavor-specific macros are known and supported by the configuration:
-
%__python2
: path to the $flavor executable. This exists mostly for Fedora compatibility. In SUSE code, it is preferable to use$flavor
directly, as it is specified to be the name in/usr/bin
, and we don't support multiple competing binaries (in the OBS environment at least). -
%python2_sitelib
,%python2_sitearch
: path to noarch and arch-dependentsite-packages
directory. -
%python2_version
: dotted major.minor version.2.7
for CPython 2.7. -
%python2_version_nodots
: concatenated major.minor version.27
for CPython 2.7. -
%python2_bin_suffix
,%python3_bin_suffix
,%pypy3_bin_suffix
: what to put after a binary name. Binaries for CPython are calledbinary-%{python_version}
, for PyPy the name isbinary-pp%{pypy3_version}
-
%python2_prefix
: prefix of the package name.python
for old-style distros,python2
for new-style. For other flavors, the value is the same as flavor name.For reasons of preferred-flavor-agnosticity, aliases
python_*
are available for all of these.We recognize
%py_ver
,%py2_ver
and%py3_ver
as deprecated spellings of%flavor_version
. No such shortcut is in place forpypy3
. Furthermore,%py2_build
,_install
and_shbang_opts
, as well aspy3
variants, are recognized for Fedora compatibility.
Files in Repository
-
macros
directory: contains a list of files that are concatenated to produce the resultingmacros.python_all
file. This directory is incomplete, files020-flavor-$flavor
and040-automagic
are generated by running the compile script. -
macros/001-alternatives
: macro definitions for alternatives handling. These are not Python-specific and might find their way intoupdate-alternatives
. -
macros/010-common-defs
: setup for macro spelling templates, common handling, preferred flavor configuration etc. -
macros/030-fallbacks
: compatibility and deprecated spellings for some macros. -
compile-macros.sh
: the compile script. Builds flavor-specific macros, Lua script definition, and concatenates all of it intomacros.python_all
. -
apply-macros.sh
: compile macros and runrpmspec
against first argument. Useful for examining what is going on with your spec file. -
flavor.in
: template for flavor-specific macros. Generatesmacros/020-flavor-$flavor
for every flavor listed incompile-macros.sh
. -
functions.lua
: Lua function definitions used inmacros.lua
and elsewhere. In the compile step, these are converted to a RPM macro%_python_definitions
, which is evaluated as part of%_python_macro_init
. This can then be called anywhere that we need a ready-made Lua environment. -
macros.in
: pure-RPM-macro definitions for the single-spec generator. References and uses the private Lua macros. The line### LUA-MACROS ###
is replaced with inlined Lua macro definitions. -
macros.lua
: Lua macro definitions for the single-spec generator.
This is actually pseudo-Lua: the top-level functions are not functions, and are instead converted to Lua macro snippets. (That means that you can't call the top-level functions from Lua. For defining pure Lua functions that won't be available as Lua macros, use thefunctions.lua
file.) -
macros-default-pythons
: macro definitions for%have_python2
and%have_python3
for systems where this is not provided by your Python installation. The spec file uses this for SUSE <= Leap 42.3.apply-macros
also uses it implicitly (for now). -
embed-macros.pl
: takes care of slash-escaping and wrapping the Lua functions and inserting them into themacros.in
file in order to generate the resulting macros. -
python-rpm-macros.spec
: spec file for thepython-rpm-macros
package generated from this GitHub repository. -
process-spec.pl
: Simple regexp-based converter into the singlespec format. -
README.md
: This file. As if you didn't know.