opensuse/obs-service-set_version

An OBS source service: Update spec file version

set_version (OBS source service)

This is an Open Build Service source service. It updates an RPM spec or Debian changelog according to the existing files.

This is the git repository for openSUSE:Tools/obs-service-set_version. The authoritative source is https://github.com/openSUSE/obs-service-set_version

The service can be used in combination with other services like download_files, tar_scm, recompress or extract_file e.g. within the GIT integration workflow.

Dependencies

Install the following deps:

zypper in python-packaging

Test suite

To run the full testsuite, some dependencies are needed:

zypper in devscripts dpkg python-flake8

If the dependencies are not installed, some tests are skipped. zypper itself is also needed for the tests with python packages and PEP440 compatible versions.

To run the full testsuite, execute:

python -m unittest discover tests/

If zypper and/or dpkg are installed, theses tests take some time, but you can specify a filename pattern, which test files should be run.

python -m unittest discover -p test_b*.py tests/

Don't forget to run also

flake8 set_version tests/

or simply use

make test

to run all linters and tests

obs-service-set_version's Issues

>Python 3.7 compability

While using the params of this service I noticed that I get a Python error.

┬─[enno@g232:~/E/h/onedrive]─[14:05:39]
╰─>$ osc -A https://api.opensuse.org service disabledrun
merge: origin/v2.4.15 - not something we can merge
Already up to date.
9c01ac3151ad6068607684f3f8ffadc996f3fc79
Detection failed with error: " module 'os' has no attribute 'errno' ".
Aborting: service call failed:  /usr/lib/obs/service/set_version --fromfile configure --regex PACKAGE_VERSION --outdir /home/enno/ExternalBuildService/home:SchoolGuy:branches:network/onedrive/tmpfndprlof.set_version.service
┬─[enno@g232:~/E/h/onedrive]─[14:05:43]
╰─>$ python3
Python 3.8.12 (default, Aug 31 2021, 01:23:42) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.errno
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'errno'
>>> exit()

This is correct behavior of Python and was published in the release notes of Python 3.7.

See: https://docs.python.org/3/whatsnew/3.7.html#changes-in-the-python-api

PKGBUILD version not updated

https://build.opensuse.org/package/show/home:skunkos/rssguard-git

Note contents of original PKGBUILD and generated one. Version was not regenerated as per _service. It worked several days ago.

if name in PARENT_TAG unable to build

For home:jubalh:gitbuilds/swift-im I used <param name="versionformat">@PARENT_TAG@~git@TAG_OFFSET@.%h</param>.
git describe sais swift-4.0beta2-17-ga0f588c.

During build I get:

[    0s] Using BUILD_ROOT=/var/tmp/build-root/openSUSE_Tumbleweed-x86_64
[    0s] Using BUILD_ARCH=x86_64:i686:i586:i486:i386
[    0s] 
[    0s] 
[    0s] hostname started "build swift-im.spec" at Tue Aug  9 11:53:58 UTC 2016.
[    0s] 
[    0s] 
[    0s] processing recipe /home/michael/obs/home:jubalh:gitbuilds/swift-im/swift-im.spec ...
[    0s] running changelog2spec --target rpm --file /home/michael/obs/home:jubalh:gitbuilds/swift-im/swift-im.spec
[    1s] cp: omitting directory '/home/michael/obs/home:jubalh:gitbuilds/swift-im/swift'
[    1s] Unpacking swift-swift4.0beta2~git17.a0f588c.obscpio ...
[    1s] 173547 blocks
[    1s] Running build time source services...
[   27s] Compressed swift-swift4.0beta2~git17.a0f588c.tar to swift-swift4.0beta2~git17.a0f588c.tar.xz
[   27s] /usr/lib/obs/service/set_version:32: RuntimeWarning: install 'packaging' to improve python package versions
[   27s]   RuntimeWarning)
[   27s] service run failed for service 'set_version'

Adrian mentioned that the following works fine thoug: <param name="versionprefix">4.0beta2~git</param>.

It seems that if the name is in @PARENT_TAG@ we can't handle it?

Don't allow arbitrary files to be modified

Original report: https://bugzilla.suse.com/show_bug.cgi?id=1143652

The service can specify an absolut/relative path. Only files in the current directory should be modifiable

Debian non-native packaging compatibility

Debian non-native packages fail to build unless the version contains a release number (i.e. ends with "-digits"), e.g. 1.2.3-4 indicating the fourth Debian release of upstream 1.2.3.

Such release numbering is incompatible with RPM.

I'm requesting the ability to define a new "debrelease" parameter that when specified is appended to the version substitution for just the Debian control files. Does this sound acceptable in which case I can perhaps send a pull request?

Enable travisCI pull request integration

After #12 was merged, PR integration in TravisCI must be enabled. I think it can be done here: https://travis-ci.org/openSUSE/obs-service-set_version/ . @adrianschroeter can you do that, please?

set_version sets wrong version

When I use osc service mr on network:vpn/tailscale, for some reason it finds the older version 1.66.1 rather than the latest version (which I download with tar_scm) 1.66.4.

Not sure where this version comes from, as no other tarball exists, or remarked anywhere (except the changes file)

OBS package should recommend python-packaging for openSUSE as well

Does not seem to run properly without.

obscpio containing zip file raises BadZipFile exception

The zipfile.is_zipfile unfortunately misclassifies any file which has a zipfile end-of-central-directory header in its last 65kByte as a zipfile:

python/cpython#72680

This is often the case when one of the last files in the obscpio archive is a zip file, or any other file which contains a zip file.

The check should probably ignore any python exceptions, or at least also BadZipFile:

obs-service-set_version/set_version

Lines 249 to 252 in d4150d2

	except OSError:
	# is_zipfile has often false positives and module is
	# crashing on processing
	pass

testsuite fails on newer distros probably python3 issue

build monitor link

[   44s] + make test PYTHON=python3
[   44s] flake8 set_version tests/
[   45s] set_version:107:24: W605 invalid escape sequence '\d'
[   45s] set_version:107:29: W605 invalid escape sequence '\/'
[   45s] set_version:158:12: W605 invalid escape sequence '\s'
[   45s] set_version:158:33: W605 invalid escape sequence '\;'
[   45s] make: *** [Makefile:14: test] Error 1

Cannot build on ARCHLINUX if there is more than one source

This service replaces the checksum with ('SKIP') even if there are multiple source files, creating an invalid PKGBUILD
error

[    2s] ==> ERROR: Integrity checks (sha256) differ in size from the source array.

What the service generates:

source=("${pkgname}-${pkgver}.tar.xz" "cargo_config" "vendor.tar.xz")
sha256sums=('SKIP')

and what (at the bare minimum) it should generate:

source=("${pkgname}-${pkgver}.tar.xz" "cargo_config" "vendor.tar.xz")
sha256sums=('SKIP' 'SKIP' 'SKIP')

"Unable to detect version"

The service should really tell me why it felt unable.

19:47 a4:../telephony/libosmo-dsp > sosc service disabledrun
Detected cached repository...
6561cfa8da6ddba35294ede452c1e5771b56e806
6561cfa8da6ddba35294ede452c1e5771b56e806
Identical target file libosmo-dsp-v0.3.8.tar.xz already exists, skipping..
unable to detect the version
Aborting: service call failed:  /usr/lib/obs/service/set_version --outdir /home/jengelh/obs/zu/network/telephony/libosmo-dsp/home/jengelh/branches/network/telephony/libosmo-dsp/tmpR4Tgzv.set_version.service
19:47 a4:../telephony/libosmo-dsp > cat _service 
<services>
        <service name="tar_scm" mode="disabled">
                <param name="scm">git</param>
                <param name="url">git://git.osmocom.org/libosmo-dsp</param>
                <param name="revision">master</param>
                <param name="versionformat">@PARENT_TAG@.@TAG_OFFSET@</param>
        </service>
        <service name="recompress" mode="disabled">
                <param name="file">*.tar</param>
                <param name="compression">xz</param>
        </service>
        <service name="set_version" mode="disabled"/>
</services>

set_version fails to detect version correctly if archive contains files with versions

For example, if archive contains:

druid-0.9.2/extensions/druid-avro-extensions/avro-1.7.7.jar

set_version will guess version "1.7.7.jar" instead of "0.9.2"

I already created PR for this: #49

set_version doesn't update version field in PKGBUILD

Example: https://build.opensuse.org/package/show/home:metakcahura/sway-git

Extended set_version script to work with #!RemoteAssetUrl provided scripts.

The idea is that aside from setting the version, it is possible to replace the typical _service file, but there is only functionality missing:

 <service name="set_version" mode="buildtime"/>

It seems it shouldn't be that complicated to add new method of VersionDetector, which would essentially run git rev-parse --verify --short HEAD and compose the version number from that.

nothing provides python-ddt, nothing provides python-packaging - Mageia 7

OBS build for Mageia 7 fails with unresolvable nothing provides python-ddt, nothing provides python-packaging
There's no python-ddt package in mageia. Any ideas how to work around this?

Arch Linux / makepkg: rel is always set 0

This service/script always sets "rel" to zero instead of reading it from pkgbuild file or the source service.

_replace_tag(filename, "pkgrel", "0")

This somewhat breaks Arch Linux packaging, because on updating a package I now have to update the version number of the source while it should only be the number of "rel".

Best would be if it would read this number from pkbuild file and increment it as long as the version of the sources remain the same.

flake8 requirement not updated for debian build spec

The dependencies for debian targets did not get updated for the flake8 requirement, resulting in a package build error.

[BUG] Generated "version" field contains newline?

This concerns dsc and spec files.

See this.

Error messages are cryptic

It drives a person debugging his/her project with only the SaaS service insane.

arch linux PKGBUILD checksum handling

Original report: https://bugzilla.suse.com/show_bug.cgi?id=1143655

PKGBUILD checksum verifications are forcefully disabled. This should be removed or the user needs to be at least warned if there's a reason why we can't do this.

Regressions: things broke with perl rewrite

After service invocation:

Name: python-neutronclient
-Version: 2.3.4.19.g9d8ccad
+Version:
Release: 0

set_version set package version to "-0"

HI,
I'm building package for p4lang-p4c , while it has worked perfectly for 2 years.
Not sure why, set_version append to changelog -0 to obs_scm-versionformat. The result is that this breaks debuild build mentioning:

dpkg-source: error: can't build with source format '3.0 (native)': native package version may not have a revision

Looking at the log shows indeed that the issue is related to p4c_20211020~9b969dd6e~1.2.2~nightly-0
It should not be -0. I tried various solution in order to change this, without luck.

Can you please let me know if there is a work around so that we can recover a changelog version such as p4c_20211020~9b969dd6e~1.2.2~nightly-0+40.1 for example (+40.1 being an example, not sure if this is a random number)

PS: I'm not sure if the issue is on set_version or more globally to obs_scm. Feel free to change the issue queue in the relevant software.

Thanks !
Frederic

set python interpreter properly

As reported here RHEL_8

[   50s] + /usr/lib/rpm/brp-python-hardlink
[   50s] + PYTHON3=/usr/libexec/platform-python
[   50s] + /usr/lib/rpm/redhat/brp-mangle-shebangs
[   50s] *** ERROR: ambiguous python shebang in /usr/lib/obs/service/set_version: #!/usr/bin/python. Change it to python3 (or python2) explicitly.
[   50s] error: Bad exit status from /var/tmp/rpm-tmp.hypdis (%install)

Not sure if there are ways to skip such checks...

set_version autodetection cannot handle version numbers that start with a non-digit

A _service file using tar_scm could create a source tarball that has for example version v1.0.

set_version's version autodetection by _get_version_via_filename() is not able to detect the version from the filename because it expects the version part to start with a digit following a dash or an underscore.

At least RPM packaging does not prohibit using such a version format.

Update version in Debtransform lines.

_replace_tag function only Updates the Version tag line (i.e. Version:)

set_version service should update the version numbers in debtransfrom-tar lines.

For example:

Debtransform-Tar: package-%version.tar.gz  
Debtransform-Tar: package-$version.tar.gz 

Should have %version or $version updated to the actual version from any previous services.

Add support to add changelog entry with version when running from osc service

It would be nice if this service would support adding an associated changelog entry when running from osc service runall

If a package has a _service file that fetches tarballs from a git repo, for example, you need to do the following:

1. run service runall
2. copy version to clipboard
3. run osc vc
4. type the line "update to version" and paste the version into the file.
5. save file and close

This could be simplified by a new option for osc service which also updates the change file(s).

set_version service incorrectly replaces version with '0.geom.out' , when using tar_scm with two sources on build.opensuse.org

Daily updated snapshots packages are built for the vsg project https://github.com/vsg-dev. Due to a change in the project (see vsg-dev/VulkanSceneGraph#784), it is necessary to use two git sources in one package, which fails on build.opensuse.org with an error message

error: Bad source: /home/abuild/rpmbuild/SOURCES/VulkanSceneGraph-0.geom.out.tar.gz: No such file or directory

because in the generated spec file the version is replaced by 0.geom.out, while in a local run it works with osc. Here the version is correctly replaced by e.g. 1.0.5+20230413+git.95ae7957.

See https://build.opensuse.org/package/show/home:rhabacker:obs-tests/test-obs-service-set_version for a corresponding test case.

I also tried obs_scm, which fails with another error.

Missing license content

This source service is labeled as GPL-2.0+ in the packaging, but there's no license file for the GPLv2, nor is the requisite header in the source files present. Can this be fixed, please?

This is necessary for me to package this source service in Fedora.

Error in building debian package

When trying to build the debian package from openSUSE:Tools/obs-service-set_version with osc build --local-package xUbuntu_22.10 the build fails with this as the last messages:

[   54s] ======================================================================
[   54s] FAIL: test_python_version_pip2rpm_10___1_7_40_svn___None_ (test_python_pip2rpm.VersionConverterTest)
[   54s] See https://www.python.org/dev/peps/pep-0440/        #examples-of-compliant-version-schemes
[   54s] ----------------------------------------------------------------------
[   54s] Traceback (most recent call last):
[   54s]   File "/usr/lib/python3/dist-packages/ddt.py", line 191, in wrapper
[   54s]     return func(self, *args, **kwargs)
[   54s]   File "/usr/src/packages/BUILD/tests/test_python_pip2rpm.py", line 129, in test_python_version_pip2rpm
[   54s]     self.assertEqual(rpm_ver, expected_ver)
[   54s] AssertionError: '1.7.40~svn' != None
[   54s] 
[   54s] ----------------------------------------------------------------------
[   54s] Ran 99 tests in 42.954s
[   54s] 
[   54s] FAILED (failures=1, skipped=20)
[   54s] make[1]: *** [Makefile:15: test] Error 1
[   54s] make[1]: Leaving directory '/usr/src/packages/BUILD'
[   54s] dh_auto_test: error: make -j1 test returned exit code 2
[   54s] make: *** [debian/rules:8: build] Error 25
[   54s] dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
[   54s] 
[   54s] linux.fritz.box failed "build debian.dsc" at Mon Mar 20 11:02:19 UTC 2023.
[   54s] 

The buildroot was: /var/tmp/build-root/xUbuntu_22.10-x86_64

port testsuite from imp.py to importlib

https://docs.python.org/3.11/library/imp.html

deprecated since 3.4 and dropped in 3.12

this e.g. breaks the package on ubuntu 24.04 and probably soon TW.

picks version from a file that does not match basename

I'm running into difficulty getting set_version to pick the version value from the right filename. My configuration is like the GIT Integration example from the docs (https://en.opensuse.org/openSUSE:Build_Service_Concept_SourceService#Example_2:_GIT_integration) except that I don't have an extract_file step, as my spec file is in the obs project and is in not in source control.

set_version is picking the nonsensical version 128.png, presumably because the project contains a file logo-128.png, although it's not clear to me how set_version is aware of that file as I would expect it to exist only in tarballs at that point.

After finding this git repo and reading set_version.service, I set a basename parameter value of orion, which should match only one of the tarballs with the version suffix, but it still finds 128.png somehow.

All of this is happening on build.opensuse.org; I don't know how to determine what version of set_version is in use there.

set_version breaks PKGBUILD with multiple sources

set_version unconditionally makes the md5sums and sha256sums keys contain a single element:

obs-service-set_version/set_version

Lines 374 to 375 in 3cd89e9

	_replace_tag(filename, "md5sums", "('SKIP')")
	_replace_tag(filename, "sha256sums", "('SKIP')")

This breaks PKGBUILDS with multiple sources, where

sha256sums=('SKIP'
            '5928e483bbba726eaa6197bfe33d05b7738e47f93450412ec78d6df5c2fd16e8')

turns into invalid syntax:

sha256sums=('SKIP')
            '5928e483bbba726eaa6197bfe33d05b7738e47f93450412ec78d6df5c2fd16e8')

Until 3cd89e9, sha256sums was not touched, and pristine PKGBUILDs based on sha256sums and 'SKIP' did work.

Affected OBS package: https://build.opensuse.org/package/show/home:dg0yt/openorienteering-mapper-unstable. I don't see a workaround.

Make set_version detect version when v0.1 style version numbers are used

When building from source the version number can be taken from git tags like this:

  <service name="tar_scm">
    <param name="scm">git</param>
    <param name="url">https://github.com/seccubus/obs_autobuild_test</param>
    <param name="filename">obs_autobuild_test</param>
    <param name="versionformat">@PARENT_TAG@.@TAG_OFFSET@</param>
    <param name="filename">obs_autobuild_test</param>
  </service>

When you create a release on GitHub it recommends that you create your tags like this: v0.1

If you tag you version like this, your build breaks with the cryptic message:

Files could not be expanded: Running /usr/lib/obs/service/set_version --outdir /lxc.tmp.29937/out

It would be great if set_version could still detect the version even if it is appened with a v.

Archlinux rebuilds not maked as updates

Archlinux has a very simple package system, and it relies on upgrades only if the version or the pkgrel changes. On the main repos, every rebuild is marked by an increase in the number of pkgrel.
set_version completely ignores this and always sets the pkgrel to zero. This makes updating certain packages a very manual process (that involves deleting the cached package first).
Would it be possible to set the pkgrel to the OBS revision number? Ideally it would be nice to have it reset to 1 every time the main version changes, but even just setting it to the OBS revision number would improve the user experience a lot.

The line in question:
https://github.com/openSUSE/obs-service-set_version/blob/master/set_version#L478

Error about "python.packaging" missing despite it being installed

Hello,
on Tumbleweed I get the following error:

Compressed release-notes-sled-15.5.20230104.tar to release-notes-sled-15.5.20230104.tar.bz2
/usr/lib/obs/service/set_version:33: RuntimeWarning: install 'packaging' to improve python package versions
  warnings.warn("install 'packaging' to improve python package versions",
At revision 22c938d10c553eaf36d1f2a3e1a889b3.

However, python310-packaging is installed:

# sudo zypper in --no-recommends python310-packaging 
Loading repository data...
Reading installed packages...
'python310-packaging' is already installed.
No update candidate for 'python310-packaging-22.0-1.1.noarch'. The highest available version is already installed.
Resolving package dependencies...
Nothing to do.

Importing packaging.version modules works, except for LegacyVersion:

Python 3.10.9 (main, Dec 08 2022, 14:49:06) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from packaging.version import parse
>>> from packaging.version import Version
>>> from packaging.version import LegacyVersion
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'LegacyVersion' from 'packaging.version' (/usr/lib/python3.10/site-packages/packaging/version.py)
>>>

Could it be related to LegacyVersion being removed in https://github.com/pypa/packaging/blob/main/CHANGELOG.rst#220---2022-12-07?