boostorg/boost-ci

Continuous Integration Toolkit for boostorg code repositories

Boost.CI

This repository contains scripts that enable continuous integration (CI) with Appveyor, Azure Pipelines, codecov.io, Coverity Scan, GitHub Actions, Drone, and Travis CI. These scripts are intended to be downloaded and used during boost repository builds to improve project quality. In most cases the scripts are self-configuring. Some integrations require additional setup actions to complete.

Boost.CI also allows you to run a big-endian build on Travis CI and Github Actions.

Build Status

GH Actions	Appveyor	Azure Pipelines	Drone	codecov.io

Summary (TL;DR)

Here are all the steps you need to take as a Boost repository maintainer to enable all of these CI features in your repository. (Note that you may skip some steps, e.g. if you don't need a specific CI service):

1. Checkout develop and then make a new branch called ci.
2. Copy the .appveyor.yml file from this repository into the top level of your repository.
3. Copy the .azure-pipelines.yml file from this repository into the top level of your repository.
4. Copy the .travis.yml file from this repository into the top level of your repository.
5. Copy the .github/workflows/ci.yml file from this repository into the the same folder in your repository.
   • Note that those include CMake tests, so see the CMake docs for details and requirements.
6. Copy the .drone.star file and optionally the .drone directory from this repository to the top level of your repository.
7. Copy the .codecov.yml file from this repository to the top level of your repository and edit if required. Note that only the file in the default branch (usually master) is considered!
8. Copy the LICENSE file from this repository to the top level of your repository. This adds the BSL-1.0 designation to your repository on github.
9. [optional] Copy the README.template.md file from this repository to the top level README.md of your repository. If you already have a README.md then you can take what you need from the template version to improve it, if desired. Otherwise, you will need to customize README.md for your repository. One useful step is to fixup the repository name using the command sed -i 's/template/<myrepositoryname>/g' README.md, and then update the first line description.
10. In Appveyor, add a project for your fork of the repository. No customization is needed.
11. In Travis CI, add a project for your fork of the repository. Later you will customize it for Coverity Scan, but for now no settings changes are necessary.
12. Commit these changes and push to your personal fork in the ci branch.
13. Create a pull request in your fork of /ci to /develop. Do not target boostorg/develop.
14. Observe that the CI services are running the build jobs. Fix up any issues found. Note this may uncover defects in your repository code.
15. If you are the owner or an admin for your repository, add projects in Appveyor and Travis CI for the boostorg/ project (not your fork). If you are just a contributor in the repository, create an issue in Boost.Admin requesting Appveyor and Travis CI to be enabled for the repository.
16. Commit the changes to develop. This will kick off a build.
17. Update the badge matrix in README.md with the correct links for your CI projects in use (e.g. Appveyor, Github Actions, Travis).
18. Create a Coverity Scan account if you have not already done so.
19. Create a new Coverity Scan github based project for your official boostorg repository.
20. Update your Travis CI and/or Github Actions boostorg repository project settings and add the following environment variables using respective CI GUI:
    • COVERITY_SCAN_NOTIFICATION_EMAIL can be public and set to your email account (or it can be private).
    • COVERITY_SCAN_TOKEN should be kept private (aka. "secret") and set to the scan token you can find in the project settings in Coverity Scan.
21. Update the README.md to put the correct Coverity Scan badge project number into the badge URLs.
22. This will kick off a build on the develop branch that will include Coverity Scan results.
23. To activate Drone, visit https://drone.cpp.al. Authorize Drone: Click the "Authorize cppalliance-drone" button. Sync repositories: Click the "sync" button. A list of repositories will appear. For the relevant repo, click and then choose "Activate Repository". In the settings page, change Configuration from .drone.yml to .drone.star. "Save".
24. More pointers about Drone:
    • Ensure that shell scripts are executable: chmod 755 .drone/drone.sh
    • The install-script (in ci/drone) and run-script (in .drone) for the Unix-ish jobs will be downloaded from Boost.CI if they don't exist, so you only need them when you want to customize the build.
    • The .drone/{before,after}-install.* scripts are sourced around the common_install step (which e.g. bootstraps B2) of the default build, if they exist. So you can remove them when not required.
    • "asan" jobs require elevated privileges. Contact an administrator or open an issue at drone-ci to set your drone repository to "Trusted".
    • If not using asan, simply remove the jobs.
    • For Codecov you need to copy the "Repository Upload Token" from the settings page of your repo on Codecov and use it to create a new secret named codecov_token on the settings page of your repo on Drone.
    • If you need a package installed on MacOS or FreeBSD, by the root user, please open an issue.
    • Further info available at https://github.com/CPPAlliance/drone-ci

Code coverage

Multiple CI configs contain jobs collecting code coverage data. E.g. Github Actions and Drone CI for Linux and Appveyor for Windows. Especially the latter allows to collect coverage data for Windows-only codeparts or code using e.g. wchar_t which is different on Windows than on other platforms.

Exclusion of coverage data

If you want to exclude parts of your code from coverage analysis you can use the LCOV comments/macros:

• // LCOV_EXCL_LINE for a single line
• // LCOV_EXCL_START and // LCOV_EXCL_STOP for a range of code

See the LCov manual for more information.

To exclude whole files or folders you can use the ignore-object and glob patterns in in the .codecov.yml file. See the example file in the root of this repository for a starting point.
Important: Codecov only considers the configuration file of the default branch (usually: master), not the one in the current build/PR branch. See the CodeCov documentation for details.

Repositories using Boost.CI

The CMT Stale Repo Tracker identifies many repositories using Boost.CI and the CMT Status Spreadsheet shows the current state of each. There may be additional repositories using Boost.CI that are not listed. Boost.CI does not track usage internally.

How It Works

The CI config files (such as .appveyor.yml, .azure-pipeline.yml and .github/workflows/ci.yml) must exist in your repository and will contain your customizations for build types, languages, and platforms. The templates provided will get you started with the build jobs listed below.

These scripts will copy resources from the Boost.CI repository when needed in order to provide scripting necessary to run all these jobs successfully.

Build jobs that will severely impact performance (such as valgrind) will define BOOST_NO_STRESS_TEST so those can be skipped or hobbled.

Topic Branch Support

The configuration for Github Actions, Azure Pipelines, Appveyor and Travis CI allow for automated branch builds on branch pushes matching these names:

• master
• develop
• bugfix/*
• feature/*
• fix/*
• pr/*

Note that when opening a Pull Request (PR) the use of a topic branch name is not required and even discouraged as PRs are always built on CI and the use of a topic branch name will lead to duplicating all builds.

A good strategy is to only use a topic branch name when actively working on a changeset and when you want CI to run for each push of that branch. When the changeset is (mostly) done it is advised to rename the branch right before opening a PR to avoid that double-build issue. This can be done easily via the GitHub website (check the "branches" link at the repo overview page) or of course via command line (push to a new remote branch and delete the old one).

Defaults, Builds and Services

By default the builds target multiple different C++ versions (from C++03 to C++20), and this can be customized. To see what kind of coverage these builds provide, see some build results as follows or click the badges above:

AppVeyor : https://ci.appveyor.com/project/Flamefire/boost-ci/branch/master
Github Actions : https://github.com/boostorg/boost-ci/actions/workflows/ci.yml
Azure Pipelines : https://dev.azure.com/boostorg/boost-ci/_build/latest?definitionId=8&branchName=master
Travis CI : https://travis-ci.org/boostorg/uuid/builds/449557162

Without any customization the scripts can provide the following services (example only, see the actual CI scripts for current configurations):

CI	description	toolset	cxxflags/std	address-model	variant
Appveyor	MSVC 2019 C++2a Strict	msvc-14.2	2a, -permissive-	64	release
Appveyor	MSVC 2017 C++2a Strict	msvc-14.1	2a, -permissive-	64	release
Appveyor	MSVC 2017 C++17	msvc-14.1	17	64	debug
Appveyor	MSVC 2017 C++17	clang-win	11	64	release
Appveyor	MSVC 2017 C++14 Default	msvc-14.1	default (14)	32,64	release
Appveyor	MSVC 2015 C++14 Default	msvc-14.0	default (14)	32,64	debug
Appveyor	MSVC 2013	msvc-12.0	default (most of 11)	default	release
Appveyor	MSVC 2012	msvc-11.0	default (some of 11)	default	release
Appveyor	MSVC 2010	msvc-10.0	default (some of 0x)	default	release
Appveyor	cygwin	gcc	03,11	32	debug
Appveyor	cygwin64	gcc	11,17	64	release
Appveyor	mingw	gcc	03,11	32	debug
Appveyor	mingw64	gcc	11,17	64	release
Azure P.	gcc 4.8	gcc-4.8	03,11	default	debug,release
Azure P.	gcc 4.9	gcc-4.9	03,11	default	debug,release
Azure P.	gcc 5	gcc-5	11	default	debug,release
Azure P.	gcc 6	gcc-6	11,14	default	debug,release
Azure P.	gcc 7	gcc-7	11,14,17	default	debug,release
Azure P.	gcc 8	gcc-8	14,17,2a	default	debug,release
Azure P.	clang-3.5	clang-3.5	03,11	default	debug,release
Azure P.	clang-3.6	clang-3.6	03,11	default	debug,release
Azure P.	clang-3.7	clang-3.7	03,11	default	debug,release
Azure P.	clang-3.8	clang-3.8	03,11,14	default	debug,release
Azure P.	clang-3.9	clang-3.8	03,11,14	default	debug,release
Azure P.	clang-4.0	clang-4.0	11,14,17	default	debug,release
Azure P.	clang-5.0	clang-5.0	11,14,17	default	debug,release
Azure P.	clang-6.0	clang-6.0	14,17,2a	default	debug,release
Azure P.	clang-6.0-libc++	clang-6.0	03,11,14,17,2a, libc++	default	debug,release
Azure P.	clang-7	clang-7	14,17,2a	default	debug,release
Azure P.	clang-8	clang-8	14,17,2a	default	debug,release
Azure P.	MSVC 2019 C++2a Strict	msvc-14.2	2a, -permissive-	64	debug,release
Azure P.	MSVC 2017 C++2a Strict	msvc-14.1	2a, -permissive-	64	debug,release
Azure P.	MSVC 2017 C++17	msvc-14.1	17	32,64	debug,release
Azure P.	MSVC 2017 C++14 Default	msvc-14.1	default (14)	32,64	debug,release
Azure P.	MSVC 2015 C++14 Default	msvc-14.0	default (14)	32,64	debug,release
Azure P.	Xcode 10.1	clang	14,17,2a	default	debug,release
Azure P.	Xcode 10.0	clang	14,17,2a	default	debug,release
Azure P.	Xcode 9.4.1	clang	11,14,17	default	debug,release
Azure P.	Xcode 9.4	clang	11,14,17	default	debug,release
Azure P.	Xcode 9.3.1	clang	11,14	default	debug,release
Azure P.	Xcode 9.3	clang	11,14	default	debug,release
Azure P.	Xcode 9.2	clang	11,14	default	debug,release
Azure P.	Xcode 9.1	clang	03,11	default	debug,release
Azure P.	Xcode 9.0.1	clang	03,11	default	debug,release
Azure P.	Xcode 9.0	clang	03,11	default	debug,release
Azure P.	Xcode 8.3.3	clang	03,11	default	debug,release
Travis CI	gcc 4.8	gcc-4.8	03,11	default	release
Travis CI	gcc 4.9	gcc-4.9	03,11	default	release
Travis CI	gcc 5	gcc-5	03,11	default	release
Travis CI	gcc 6	gcc-6	11,14	default	release
Travis CI	gcc 7	gcc-7	14,17	default	release
Travis CI	gcc 8	gcc-8	17,2a	default	release
Travis CI	gcc 9	gcc-9	17,2a	default	release
Travis CI	clang-3.8	clang-3.8	03,11	default	release
Travis CI	clang-4.0	clang-4.0	11,14	default	release
Travis CI	clang-5.0	clang-5.0	11,14	default	release
Travis CI	clang-6.0	clang-6.0	14,17	default	release
Travis CI	clang-6.0-libc++	clang-6.0	03,11,14, libc++	default	release
Travis CI	clang-7	clang-7	17,2a	default	release
Travis CI	clang-8	clang-8	17,2a	default	release
Travis CI	osx (clang)	clang	03,11,17	default	release
Travis CI	big-endian	gcc	default	default	debug
Travis CI	codecov.io	gcc-8	default	default	debug
Travis CI	covscan	clang	default	default	debug
Travis CI	asan	gcc-8	03,11,14	default	debug
Travis CI	tsan	gcc-8	03,11,14	default	debug
Travis CI	ubsan	gcc-8	03,11,14	default	debug
Travis CI	valgrind	clang-6.0	03,11,14	default	debug

boost-ci's Issues

Travis Ubuntu 20.04

Travis supports also Ubuntu 20.04, see https://docs.travis-ci.com/user/reference/focal/ which seems to be not part of travis.yml yet

could not read Username for https://github.com

When I update Boost.JSON to use the new .azure-pipelines.yml, I get failures:
https://vinniefalco.visualstudio.com/json/_build/results?buildId=769&view=logs&j=eaa15c43-7f79-57f7-ec56-5d5933cacfc8&t=cb081657-b69e-52d4-18c8-e97eff82a2e9&l=28

The older template works:
https://github.com/vinniefalco/json/blob/4c03bec97c57e895b9d0582f3f34ea3308a1243f/.azure-pipelines.yml

Use sanitizer features from Boost.Build

Boost.Build now provides common features for the sanitizers:
boostorg/build#409

Shall we switch the COMMENT=ubsan to use those?

Second, I'd suggest to split the single COMMENT=ubsan job into job per sanitizer type (undefined, nullability, integer).
For example, in Boost.GIL, the two first pass but the -fsanitize=integer failed.

Support for CI via github actions

Are there plans to add support for CI builds via github actions? Should be similar to the azure pipeline yml, see https://github.com/actions/virtual-environments

Remove cppcheck

The installation takes time and fails often (https://travis-ci.org/boostorg/signals2/jobs/542405347), and nobody seems to be using it; we have UBSAN, valgrind, and Coverity Scan on the builds. I think we should disable the cppcheck build and update the template to remove it as a job.

Thoughts on why codecov shasum fails in CI but not locally?

The SHASUM of the codecov executable fails SHASUM check in GHA but not locally and apparently not in appveyor, so I am confused.

I added as no-cache... I suspect it is getting cached somewhere, and the cache is ignoring pragma no-cache, or there's something else going on.

https://github.com/boostorg/boost-ci/runs/5093231630?check_suite_focus=true#step:7:172

Split coverage collecting into setup and upload phase and do a simple build inbetween

I'd like to change the coverage collecting a bit and want some feedback @mloskot @pdimov if you like

Coverage is a 3 step process:

• Set up B2_CXXFLAGS etc.
• Build and run tests
• Postprocess and upload coverage reports

Some libraries have different configurations that they want to test. So I already split that process into 2 steps (prior all 3 was one script call) into "collect" (step 1&2) and "upload" (step 3) to be able to run collect multiple times before uploading.

I think this is still an issue. Now we have to duplicate the collect step and the regular build step. Because obviously the different configurations should be tested for coverage and non-coverage builds. I.e. the coverage build step should be the same as the regular build step. So we should only have

• Set up B2_CXXFLAGS etc.
• Regular build step
• Upload step

I.e. change

boost-ci/.github/workflows/ci.yml

Lines 165 to 171 in f19a988

	- name: Run tests
	if: '!matrix.cmake_tests && !matrix.coverage'
	run: ci/build.sh
	- name: Run tests with coverage collection
	if: matrix.coverage
	run: ci/codecov.sh "collect"

to this (more or less): https://github.com/boostorg/nowide/blob/5c762e889ad08bcc15a2c8000a38ffe9a1e8cc9c/.github/workflows/posix.yml#L86-L91

Note: Actually this https://github.com/boostorg/nowide/blob/5c762e889ad08bcc15a2c8000a38ffe9a1e8cc9c/.github/workflows/posix.yml#L93-L97 should not be conditioned on coverage...

So basically: codecov.sh should NOT call build.sh but simply set up the env vars/flags so the regular build.sh can be used, i.e. this:

boost-ci/.github/workflows/ci.yml

Lines 165 to 167 in f19a988

	- name: Run tests
	if: '!matrix.cmake_tests && !matrix.coverage'
	run: ci/build.sh

To be backwards compatible: The travis codecov.sh (only one we had so far) can simply call the main codecov.sh with setup and then do the build but in the template CI scripts we do it "properly"

AppVeyor configuration prompting for overwrite some file(s)

I just configured my project to use AppVeyor:
https://github.com/lpranam/astronomy/blob/CI/appveyor.yml

I encountered this bug:
https://ci.appveyor.com/project/lpranam/astronomy/builds/25792430/job/1r5xckyfoe6090g7

It prompts the message in the console and then it has to wait for infinite time as no input cannot be given and ultimately job fails due to the time limit.

I am not sure if I am doing anything wrong or it is indeed a bug. I am not much familiar CIs.

Enable GitHub Discussions?

Any objections to enable the Discussions for asking questions, discussing stuff or you'd rather prefer Issues for 'end-user' questions?

Not assume repository name is the same as library name

I encountered this when I was testing my fork of dynamic_bitset. My fork wasn't glenfe/dynamic_bitset, but instead glenfe/boost.dynamic_bitset.

Users may fork a Boost library into a repository name that isn't the same as the Boost library name. The Travis and Appveyor scripts you provide should not make the contrary assumption.

Boost libraries like core, smart_ptr, align, etc. that do not use boost-ci for Travis and Appveyor do not suffer from this problem.

Use TRAVIS_COMPILER

Travis supports compiler: foobar as a key for jobs and sets TRAVIS_COMPILER to that

I suggest:

• Use compiler instead of setting B2_TOOLSET in .travis.yml
• If B2_TOOLSET is unset but TRAVIS_COMPILER is set then set toolset from TRAVIS_COMPILER and put that into project_settings: echo "using $B2_TOOLSET : : $TRAVIS_COMPILER ;" > ~/user-config.jam

Note: TRAVIS_COMPILER may be /usr/bin/clang++. In that case B2_TOOLSET must be set to clang. Similar for clang++-3.3 --> clang-3.3

My current code:

    if [[ "$TRAVIS_COMPILER" =~ clang ]]; then
        export B2_TOOLSET=clang
    elif [[ "$TRAVIS_COMPILER" =~ g\+\+ ]]; then
        export B2_TOOLSET=gcc
    else
        false
    fi

Add MacOS 14 / IOS 17 with xCode 15.0.1 to Drone

Apparently there are changes to e.g. libiconv in recent MacOs and/or xcode. See e.g. boostorg/locale#206

So I'd like to at least have CI for those.

Is this possible for Drone?

Drone FreeBSD failures (warnings, segfault, missing _ZTIDu)

I have trouble with FreeBSD on Drone and would need a hand:

• job(compiler='clang-10', cxxstd='11,14,17,20', os='freebsd-13.1') works but throws warnings for system headers in /usr/local
• job(compiler='clang-15', cxxstd='11,14,17,20', os='freebsd-13.1') fails at runtime with Undefined symbol "_ZTIDu"
• job(compiler='gcc-11', cxxstd='11,14,17,20', os='freebsd-13.1', linkflags='-Wl,-rpath=/usr/local/lib/gcc11') fails at runtime with a seg fault

Any idea how to solve or even approach this without a local FreeBSD?

Wrong compiler used on Azure

Check https://dev.azure.com/boostorg/boost-ci/_build/results?buildId=1511&view=logs&j=288c7346-de5f-5112-54bc-d1c2ae665540&t=528f5383-0ab2-5f0a-4b8f-7e7b11d5464f
The clang 3.7 build has some issues which make Boost.CI fallback to clang++ which then uses the system clang 9!

Trying to find out where that clang is

Output:

++ command -v clang++-3.7
++ echo 'WARNING: Compiler clang++-3.7 was not installed properly'
WARNING: Compiler clang++-3.7 was not installed properly
+++ command -v clang++-3.7
++ echo 'using clang : :  :  ;'

Improve matrix readability AppVeyor

This is a trivial 'feature' request I'd like to propose.

Would it be acceptable to 'hide' the non-essential and redundant, for reader of the build matrix on AppVeyor, tag APPVEYOR_BUILD_WORKER_IMAGE and bring the more important build properties to the front?

To turn this

    - FLAVOR: Visual Studio 2017 C++2a Strict
      APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
      B2_ADDRESS_MODEL: address-model=64
      B2_CXXFLAGS: cxxflags=-permissive-
      B2_CXXSTD: latest # 2a
      B2_TOOLSET: msvc-14.1

into this:

    - FLAVOR: Visual Studio 2017 C++2a Strict
      B2_ADDRESS_MODEL: address-model=64
      B2_CXXFLAGS: cxxflags=-permissive-
      B2_CXXSTD: latest # 2a
      B2_TOOLSET: msvc-14.1
      APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017

for all the matrix jobs.

CMake tests assume cmake_subdir_test and cmake_install_test folders are present

The CMake tests assume that $SELF/test/cmake_subdir_test and $SELF/test/cmake_install_test are present in the repo and have the right CMakeLists.txt in them, like in https://github.com/boostorg/variant2.

Please add a note to installation instructions here in the README that one has to adapt the CMake builds when porting to boost-ci.

CMake install tests failing when using dependend libraries

I tried Boost.CI on Boost.Locale where a test binary links against Boost::locale which privately links against Boost::thread.

This fails on some platforms, e.g. Ubuntu 20.04 as the Boost.Thread library cannot be found. See e.g. https://github.com/Flamefire/locale/runs/6047122561?check_suite_focus=true

/home/runner/work/locale/boost-root/libs/locale/test/cmake_test/build_cmake_install_test/main: error while loading shared libraries: libboost_thread.so.1.80.0: cannot open shared object file: No such file or directory

But the library is installed:

Installing: /home/runner/.local/lib/libboost_thread.so.1.80.0

Also on Windows the test also fails likely due to the same reason:

1/1 Test #1: main .............................Exit code 0xc0000135
***Exception: 0.01 sec

It looks like we'd need to add the library install path to LD_LIBRARY_PATH on Linux and something similar on Windows (PATH?) although I feel like this is something which should be handled by CMake but isn't.

Would be good if we could fix that "by default" in the Boost.CI samples.

@pdimov Any ideas?

Add MSYS2 to Appveyor tests

We test with MinGW and Cygwin but not with MSYS2.

I'm not really sure about the difference of MinGW vs MSYS2 MinGW (maybe someone can help here?) but might be worth adding. Especially as it seems that the MSYS2 MinGW environment has e.g. GCC 11 and I have a bugreport which sounds like this environment was used with (some?) Boost libraries.

Use valid jobs names for AzP

The names (keys) in the jobs.matrix entries are invalid:

Matrix configuration names must contain only basic Latin alphabet letters (A-Z and a-z), digits (0-9), and underscores (_). They must start with a letter. Also, their length must be 100 characters or fewer.

https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#job

We should replace the spaces by underscores

Add macos-10.14 with Xcode 11 to Azure Pipelines configuration

Currently, AzP configuration is based on macos-10.13 and Xcode 8.x through 10.x

https://github.com/microsoft/azure-pipelines-image-generation/blob/master/images/macos/macos-10.13-Readme.md

Apparently, AzP offers macOS 10.14 with multiple Xcode 11.x versions, but there are no Xcode 8.x and earlier 9.x
https://github.com/microsoft/azure-pipelines-image-generation/blob/master/images/macos/macos-10.14-Readme.md

It may be a good idea to cover both macOS 10.13 and 10.14 with full coverage of Xcode versions.

Coverity details

We need someone with access to the coverity project of boost-ci to add the details (token and email) to Drone to fix the failure.

This went unnoticed so far due to a bug which made the script never execute that i just fixed.

But no idea who the admin there is that can grant access.

Avoid repetitions like B2_LINKFLAGS=linkflags=foo

Can we please avoid these repetitions of name and flag? I guess we need a new B2_FLAGS to pass arbitrary flags as e.g. currently B2_CXXFLAGS is abused to pass flags to b2 like ASAN.

@pdimov uses ${LINKFLAGS:+linkflags=$LINKFLAGS} and ${VISIBILITY:+visibility=$VISIBILITY} which I find pretty neat :)

Switch from jeking3/bdde multiarch to arch: s390x for Travis big-endian testing

Travis added a s390x architecture target in November 2019 which allows for big-endian testing, eliminating the need for multiarch docker container testing. The example should be updated.

Maintaining single address-model per build job on Windows

Why some of MSVC jobs specify both address-model-s, i.e. 32 and 64, e.g.

boost-ci/templates/appveyor.yml

Lines 73 to 75 in ceb7055

	- FLAVOR: Visual Studio 2017 C++14 (Default)
	APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
	B2_ADDRESS_MODEL: address-model=64,32

while other only 64

boost-ci/templates/appveyor.yml

Lines 66 to 68 in ceb7055

	- FLAVOR: Visual Studio 2017 C++17
	APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
	B2_ADDRESS_MODEL: address-model=64

---

I found that multi-address-model jobs on Windows are tricky to handle if a library tests need external dependencies installed with vcpkg or conan. For example, this command does not allow to point to vcpkg-installed dependencies for both targets:

b2 address-model=64,32 include=%VCPKG_INC% library-path=%VCPKG_LIB%

---

Would it be sensible to run single address-model per build job by default? What are pros/cons?

updating boost.unordered with new boost-ci: a field-report

Last week, I started integrating the new boost-ci with boost.unordered.
At that point, boost.unordered's tests were mostly failing and for that reason there hadn't been a PR for the new drone-ci. So, I started out on my own fork of boost.unordered and Sam Darwin enabled my account to run tests with drone. Here's a short protocol of what I did and how it worked out:

• integrating boost-ci with boost.unordered
  • the docs for integrating the new ci are easy to follow
  • the only missing point is: set executable-flag for .drone/drone.sh in git
  • after that, Travis, AppVeyor and Drone were triggered to run the tests
• since some days Travis-CI rejects the builds, and I don't know why
• AppVeyor: VisualStudio-builds work, but it has troubles setting up the mingw environments (#75)
• Drone works mostly
  • the Address Sanitizer (asan) refuses to run for my fork, as it is considered untrusted
  • Valgrind sometimes is killed due to time limitation (time limit seems to be 1 hour)
  • drone.star instrumentation
    • problems with clang-9, 10 and 11 with boost.unordered (I had to disable clang-10)
    • in clang configs: changed the std-library version to match the clang version
    • added clang-12 config, with std-library-version 11 (std-library-version 12 doesn't seem to exists - I'm not a clang expert!)

Summary:
boost-ci integrates seamless with other boost projects and works out of the box. The upcoming problems originated from the CI-platforms (Travis refusing, Appveyor cannot install MinGW) and the failing tests in the boost-project (buggy compilers / test-code).

Suspicious B2_TOOLSET=03,11 in codecov.io job

This def1aee did this:

-        - B2_TOOLSET=gcc-7
+        - B2_TOOLSET=03,11
+        - B2_TOOLSET=gcc-8

but wasn't it supposed to do this?

-        - B2_TOOLSET=gcc-7
+        - B2_CXXSTD=03,11
+        - B2_TOOLSET=gcc-8

Bootstrap is not invoked with a toolset so it uses "whatever"

Let's say you are building mingw on windows with gcc; that job currently uses msvc to build the bootstrap in CI (though #137 resolves it, there are more things we need to do...).

Similarly you might have gcc and clang installed, however boost build is built with gcc in clang jobs!
See: https://github.com/boostorg/boost-ci/runs/5071619375?check_suite_focus=true

Another one is mingw; you can specify mingw to bootstrap but not to b2.

We need the bootstrap to take an argument for the toolset. Now here's one of the problems. The definition of TOOLSET that we pass to boost build is not recognized by bootstrap (from an Azure Devops job):

D:\a\1\boost-root>cmd /c bootstrap msvc-14.3 
Building Boost.Build engine
###
### "Unknown toolset: msvc-14.3"
###
### You can specify the toolset as the argument, i.e.:
###     .\build.bat msvc
###
### Toolsets supported by this script are: borland, como, gcc,
###     gcc-nocygwin, intel-win32, mingw,
###     vc12, vc14, vc141, vc142, vc143
###
### If you have Visual Studio 2017 installed you will need to either update
### the Visual Studio 2017 installer or run from VS 2017 Command Prompt
### as we where unable to detect your toolset installation.
###

CCache cache not saved

Looks like the ccache action doesn't actually save the cache. This is from the post-step:

Cache hit occurred on the primary key ubuntu-20.04--gcc-11, not saving cache.

So it doesn't update the cache! O.o

We could use a dedicated ccache action like https://github.com/hendrikmuhs/ccache-action which handles the key-creation etc. for us or do it manually (see fix_ccache branch which I'm testing right now)

branch name comparison on AzP is problematic when slashes are present

See microsoft/azure-pipelines-agent#1631

Test expression is e.g., ++ '[' azp-outdated-images == master ']' rather than fix/azp-outdated-images so if I do a PR with a / in it and it is fix/master it'll not do what we want ^^

From

boost-ci/ci/azure-pipelines/install.sh

Line 62 in 8b32c09

if [ "$BUILD_SOURCEBRANCHNAME" == "master" ]; then

They claim that there is nothing to fix, suggestion microsoft/azure-pipelines-agent#1631 (comment) is to use the full one and remove refs/heads from the front (or in this case could also check if it is refs/head/master).

Low priority, couple of ways to fix, wanted to raise issue but won't fix until the other one is done.

cxxstd=03 jobs must be done in trusty

I've been experiencing some strange behaviors on the xenial distributions in travis when building with gcc 4.8 and 4.9. The root cause is that the libstdc++6 library comes from gcc 5.4.0 on this distribution using the newer dual ABI, therefore one cannot build with gcc 4.8 or 4.9 and expect to get a good result. For example in the date_time unit tests for input facets, the code catches std::ios_base::failure, however this was not happening. With some debugging, I found what's being thrown is a std::ios_base::failure[cxx11] instead.

The fix is likely to change any 03 jobs to use trusty instead of xenial.

latest lcov

FYI - when testing against the master branch of lcov.

The new lcov flag --ignore-errors unused was needed, including in codecov.sh.

An apt package libcapture-tiny-perl was required.

The apt packages libdatetime-perl libdatetime-format-dateparse-perl were required, if running genhtml.

There are new features in lcov. To be able to use the master branch now, I wonder if both the version and the command line flags of lcov should be adjustable via variables in codecov.sh.

Pass --abbreviate-paths to b2 on Windows by default

On Windows, it is not uncommon Boost tests builds fail with error failed to write output file, e.g.

failed to write output file 
  'bin.v2\libs\gil\test\extension\dynamic_image\h-extension-dynamic_image-image_view_factory.test\msvc-14.2\release\address-model-64\asynch-exceptions-on\cxxstd-latest-iso\threading-multi\h-extension-dynamic_image-image_view_factory.obj.rsp'
!

Question: would it be sensible to run b2 with --abbreviate-paths by default on all Windows jobs?

Remove Travis CI configuration?

Shall we remove the Travis CI scripts from this repo?

It looks like none of the Boost repositories have used Travis recently
https://app.travis-ci.com/github/boostorg

Sort compilers consistently

We currently have compiler versions sorted ascending on GHA and descending on Azure.

We should decide for one scheme and change one or the other file especially with the new syntax which allows to remove redundant jobs from either one.

Question: Which one do you prefer? Ascending for me.

Begin using Travis CI Shared Build to consolidate boilerplate code

Today the travis yaml file has to be copied from a template in this repository into a file in every repository that wants to use it. A bunch of the content is boilerplate and can now be delivered by versioning this repository and using their Shared Build feature, allowing parts of travis.yml to be imported.

The net effect of rolling this out is that updates to some travis functionality can be done quickly through a version bump in a consuming repository's travis yaml, or if the consuming repository wants to make it automatic they could pull a "latest" tag that we maintain and roll forward.

In the end someone does have to make a change on each repository. Teaching dependabot to attempt automatic updates to these import versions would be great but since GitHub acquired DependaBot, it's unlikely they would do anything to support Travis.

/proc/cpuinfo does not exist on OSX

On AzP OSX image the enforce script still tries to read /proc/cpuinfo leading to:

cat: /proc/cpuinfo: No such file or directory
+++ cpus=' 0'
+++ export B2_JOBS=1
+++ B2_JOBS=1

We should at least add a fallback like sysctl -n hw.ncpu or very fancy python -c 'import multiprocessing as mp; print(mp.cpu_count())' (from https://stackoverflow.com/questions/1715580/how-to-discover-number-of-logical-cores-on-mac-os-x)

Export UBSAN_OPTIONS=print_stacktrace=1 by default

If UBSAN_OPTIONS isn't set, it should be set to print_stacktrace=1 in build.sh. This avoids the current inconsistency in .travis.yml and is usually (always?) what is wanted

mingw on AppVeyor failing - can we do anything about it?

Currently the mingw builds on AppVeyor are failing. The reason is that the package manager tries to download old images which seem no longer available (cf. below or https://ci.appveyor.com/project/tobias-loew/unordered/builds/39210079/job/pon5fdeju46656t9)
E.g. the oldest version on http://repo.msys2.org/mingw/x86_64/ is mingw-w64-x86_64-icu-66.1-1-any.
Can we do anything about it?

C:\boost-root>FOR %a IN ("gcc" "icu" "libiconv" "openssl" "xz" "zlib") DO (c:\msys64\usr\bin\env MSYSTEM=MINGW32                           c:\msys64\usr\bin\bash -l -c       "pacman --sync --needed --noconfirm mingw32/mingw-w64-i686-%a"   || EXIT /B 1 ) 
C:\boost-root>(c:\msys64\usr\bin\env MSYSTEM=MINGW32                           c:\msys64\usr\bin\bash -l -c       "pacman --sync --needed --noconfirm mingw32/mingw-w64-i686-"gcc""   || EXIT /B 1 ) 
warning: mingw-w64-i686-gcc-9.1.0-3 is up to date -- skipping
 there is nothing to do
C:\boost-root>(c:\msys64\usr\bin\env MSYSTEM=MINGW32                           c:\msys64\usr\bin\bash -l -c       "pacman --sync --needed --noconfirm mingw32/mingw-w64-i686-"icu""   || EXIT /B 1 ) 
resolving dependencies...
looking for conflicting packages...
Packages (1) mingw-w64-i686-icu-64.2-1
Total Download Size:   17.45 MiB
Total Installed Size:  93.00 MiB
:: Proceed with installation? [Y/n] warning: no /var/cache/pacman/pkg/ cache exists, creating...
:: Retrieving packages...
error: failed retrieving file 'mingw-w64-i686-icu-64.2-1-any.pkg.tar.xz' from repo.msys2.org : The requested URL returned error: 404
error: failed retrieving file 'mingw-w64-i686-icu-64.2-1-any.pkg.tar.xz' from sourceforge.net : The requested URL returned error: 404
error: failed retrieving file 'mingw-w64-i686-icu-64.2-1-any.pkg.tar.xz' from www2.futureware.at : The requested URL returned error: 404
error: failed retrieving file 'mingw-w64-i686-icu-64.2-1-any.pkg.tar.xz' from mirror.yandex.ru : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded.
C:\boost-root\libs\unordered>set lastexitcode=1 
C:\boost-root\libs\unordered>set  1>C:\Users\appveyor\AppData\Local\Temp\1\tmp6BA.tmp 
C:\boost-root\libs\unordered>echo C:\boost-root\libs\unordered  1>C:\Users\appveyor\AppData\Local\Temp\1\tmp6BB.tmp 
C:\boost-root\libs\unordered>exit /b 1 
Command exited with code 1

Multiple self-updates of B2_... variables results in error:

The self-updating of B2_...-variables in build.bat and enforce.sh leads to problems when e.g. B2_CXXFLAGS is defined in the drone.star "matrix". Then B2_CXXFLAGS gets updated twice:

in drone.star: 'B2_CXXFLAGS': '-Wno-variadic-macros'

1. script run:
   B2_CXXFLAGS=cxxflags=-Wno-variadic-macros

2.script run:
B2_CXXFLAGS=cxxflags=cxxflags=-Wno-variadic-macros

Refactor drone CI integration

The readme says:

Copy the .drone.star file and .drone directory from this repository to the top level of your repository.

So what will happen is that e.g. linux_cxx will Pull down Boost.CI:

boost-ci/ci/drone/functions.star

Line 61 in 4ed2aa3

"BOOST_CI_ORG=boostorg BOOST_CI_BRANCH=master && curl -s -S --retry 10 -L -o $BOOST_CI_BRANCH.tar.gz https://github.com/$BOOST_CI_ORG/boost-ci/archive/$BOOST_CI_BRANCH.tar.gz && tar -xvf $BOOST_CI_BRANCH.tar.gz && mv boost-ci-$BOOST_CI_BRANCH .drone/boost-ci && rm $BOOST_CI_BRANCH.tar.gz",

And the current example drone file will do that again:

boost-ci/.drone/drone.sh

Line 21 in 4ed2aa3

git clone https://github.com/boostorg/boost-ci.git boost-ci-cloned --depth 1

So this duplicates work wasting resources.

Furthermore the Drone config is much less readable than e.g. the Github config

@sdarwin Could you work on the first part, i.e. that it downloads Boost.CI at most once and only if not running on Boost.CI? I don't understand why we would need .drone/boost-ci and ci when having the latter is enough (see e.g. Github actions)

I started work on an improved drone config in a branch

AWS-hosted runners

There are multiple ways to add AWS-hosted runners to boostorg/boost-ci.

1. Update .github/workflows/ci.yml so that it uses the new runners instead of GitHub runners.

or

2. Add another file .github/workflows/ci-aws-hosted.yml This offers a choice, and doesn't force the new method. In the short term this is probably the way to go, because the upstream terraform project is in flux.

or

3. An open pull request "feat: Adding runner_enable_default_labels variable #3491" in https://github.com/philips-labs/terraform-aws-github-runner should be included in the design, and when it's available that will change the instructions for boost libraries. Therefore, wait before continuing.

In the meantime anyone is welcome to experiment with the self-hosted runners https://github.com/cppalliance/githubactions however the config will need to be modified.

Should not require root Jamfile

The scripts perform "b2 ." which requires a root Jamfile. Instead, it should perform "b2 test". Root Jamfiles should not be required to use boost-ci.

OSX 10.13 is being removed from azure

At the end of March 2020.

Provide easy customization points for drone

To make using the CI as easy as possible the common Boost build / Boost library test should be made as simple as possible.
E.g. I created a function which can be used as job(compiler='clang-3.5', cxxstd='11', os='ubuntu-16.04'), similar to the GithubActions config

What we need to do is:

• install packages (at least compiler, maybe user-defined additional packages)
• clone Boost.CI and get its ci folder
• setup Boost (i.e. common_install.sh)
• build / test (e.g. build.sh)

The first step is currently done by linux-cxx-install.sh due to not having enough space to do it in the Drone config file as on GHA.

The other 3 steps are wrapped in the drone.sh script.

This is already nice as Boost authors (with #186) only need to copy the drone.star file and adjust that for the "basic" workflow.

However e.g. for Boost.Locale I need only a slightly different install process: I need to install/setup some system locales for Linux

For GHA this is easy as I can simply add a new step into the config. For Drone I would need to copy and modify the linux-cxx-install.sh script and hence keep (at least) 2 files up to date (the star-file and the install-script)

Hence what I'd rather like to do is have a custom install script for drone which is run in addition to the default.

Ideas:

1. Clone the Boost.CI repo early, i.e. in functions.star, so I can refer to the script in the custom script
2. Make the install script check for and execute additional user-provided scripts, maybe called linux-cxx-install-pre.sh/linux-cxx-install-post.sh
3. Provide at least some convenience functions for downloading files from inside a user script, similar to https://github.com/boostorg/boost-ci/blob/master/ci/drone/functions.star#L13-L17 (e.g. source a script exporting a function inside the functions.star prior to calling the other scripts)

@sdarwin thoughts?

Reduce overall complexity of builds

Suggest we limit the number of cxxstd per build job to 2, except for some of the special build jobs (like ubsan). I have this set up in a uuid dry-run pr on my fork:

jeking3/uuid#21

Suggest we limit the variant to debug except for a couple jobs where we build release. Most projects do not have debug vs. release specific preprocessor branches, and release build optimization is expensive. Given all the other checks and balances we have, this will reduce the burden on the CI providers and get builds through faster, which everyone will enjoy.

Remove .drone-folder?

@sdarwin What is the purpose of https://github.com/boostorg/boost-ci/blob/master/.drone/drone.sh ?

The drone config does a load("@boost_ci//ci/drone/:functions.star", "linux_cxx","windows_cxx","osx_cxx","freebsd_cxx") so IMO requiring library authors to add the .drone folder provides no value, does it?

git clone --depth 1

Consider

git clone --depth 1 https://github.com/boostorg/boost-ci.git C:\boost-ci

I think this also works in the yml

clone_depth: 1

Tags and Releases

We should consider tagging and/or releasing boost-ci. When someone integrates with boost-ci they can pull it using a stable tag that will never change. That way changes we make to boost-ci will not break numbers of repositories. Thoughts?

Copy of codecov.yml not required

Currently boost-ci copies the codecov.yml contained at https://github.com/boostorg/boost-ci/blob/master/.codecov.yml to the repo root. That has multiple problems:

It is documented that the file must not be named .codecov.yml: https://docs.codecov.io/docs/codecov-yaml#section-can-i-name-the-file-codecov-yml. However the bash uploader does kinda work because it does git ls-files "*codecov.yml" "*codecov.yaml". But due to the second problem this might not be enough.

The file currently contains fixes, but those are used server-side not client side. The bash upload simple pass the LOCATION of the yaml file to the uploaded file :

# Append git file list
# write discovered yaml location
echo "$yaml" >> $upload_file

From https://codecov.io/bash

--> We can get rid of that file as it seems to work nonetheless

Clang ASAN failure with Clang 14 + libc++

In Boost.unordered and Boost.Locale I see failures after using the updated compilers from a60085a

E.g.: https://github.com/boostorg/locale/runs/6744070853?check_suite_focus=true

==21529==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) on 0x60b0000016f0
    #0 0x55f6f62020c2 in free (/home/runner/work/locale/boost-root/bin.v2/libs/locale/test/test_std_formatting.test/clang-linux-14/release/address-sanitizer-norecover/cxxstd-20-iso/link-static/stdlib-libc++/threading-multi/undefined-sanitizer-norecover/visibility-hidden/test_std_formatting+0x1320c2) (BuildId: 4876f9a0c6647775b7d8244ddeeca2157e74a2e5)
    #1 0x7f9a9707624d in std::range_error::~range_error() (/lib/x86_64-linux-gnu/libc++abi.so.1+0x2424d) (BuildId: 92148758f185dd2a38d85fe5fe008fe28543e10c)
    #2 0x7f9a97078553 in __cxa_end_catch (/lib/x86_64-linux-gnu/libc++abi.so.1+0x26553) (BuildId: 92148758f185dd2a38d85fe5fe008fe28543e10c)
    #3 0x55f6f623f892 in has_std_locale(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/home/runner/work/locale/boost-root/bin.v2/libs/locale/test/test_std_formatting.test/clang-linux-14/release/address-sanitizer-norecover/cxxstd-20-iso/link-static/stdlib-libc++/threading-multi/undefined-sanitizer-norecover/visibility-hidden/test_std_formatting+0x16f892) (BuildId: 4876f9a0c6647775b7d8244ddeeca2157e74a2e5)
    #4 0x55f6f62405d4 in main (/home/runner/work/locale/boost-root/bin.v2/libs/locale/test/test_std_formatting.test/clang-linux-14/release/address-sanitizer-norecover/cxxstd-20-iso/link-static/stdlib-libc++/threading-multi/undefined-sanitizer-norecover/visibility-hidden/test_std_formatting+0x1705d4) (BuildId: 4876f9a0c6647775b7d8244ddeeca2157e74a2e5)
    #5 0x7f9a96d4ad8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 89c3cb85f9e55046776471fed05ec441581d1969)
    #6 0x7f9a96d4ae3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 89c3cb85f9e55046776471fed05ec441581d1969)
    #7 0x55f6f617f524 in _start (/home/runner/work/locale/boost-root/bin.v2/libs/locale/test/test_std_formatting.test/clang-linux-14/release/address-sanitizer-norecover/cxxstd-20-iso/link-static/stdlib-libc++/threading-multi/undefined-sanitizer-norecover/visibility-hidden/test_std_formatting+0xaf524) (BuildId: 4876f9a0c6647775b7d8244ddeeca2157e74a2e5)
0x60b0000016f0 is located 0 bytes inside of 101-byte region [0x60b0000016f0,0x60b000001755)
allocated by thread T0 here:
    #0 0x55f6f623d13d in operator new(unsigned long) (/home/runner/work/locale/boost-root/bin.v2/libs/locale/test/test_std_formatting.test/clang-linux-14/release/address-sanitizer-norecover/cxxstd-20-iso/link-static/stdlib-libc++/threading-multi/undefined-sanitizer-norecover/visibility-hidden/test_std_formatting+0x16d13d) (BuildId: 4876f9a0c6647775b7d8244ddeeca2157e74a2e5)
    #1 0x7f9a970e1f6e in std::runtime_error::runtime_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/lib/x86_64-linux-gnu/libc++.so.1+0x4ef6e) (BuildId: b21b5b84d38618ac5758baefbe063160913da4cb)
SUMMARY: AddressSanitizer: alloc-dealloc-mismatch (/home/runner/work/locale/boost-root/bin.v2/libs/locale/test/test_std_formatting.test/clang-linux-14/release/address-sanitizer-norecover/cxxstd-20-iso/link-static/stdlib-libc++/threading-multi/undefined-sanitizer-norecover/visibility-hidden/test_std_formatting+0x1320c2) (BuildId: 4876f9a0c6647775b7d8244ddeeca2157e74a2e5) in free
==21529==HINT: if you don't care about these errors you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
==21529==ABORTING

I.e. the culprit is this job:

boost-ci/.github/workflows/ci.yml

Lines 86 to 87 in 699bb3b

	- { name: Clang w/ sanitizers, sanitize: yes,
	compiler: clang-14, cxxstd: '03,11,14,17,20', os: ubuntu-22.04, stdlib: libc++, install: 'clang-14 libc++-14-dev libc++abi-14-dev' }

We could handle that by NOT using libc++, but we have GCC 12 with libstdc++ and sanitizers already at

boost-ci/.github/workflows/ci.yml

Lines 59 to 60 in 699bb3b

	- { name: GCC w/ sanitizers, sanitize: yes,
	compiler: gcc-12, cxxstd: '03,11,14,17,20', os: ubuntu-22.04 }

Or revert back to Clang 12 like it was at

boost-ci/.github/workflows/ci.yml

Lines 83 to 84 in 76211c3

	- { name: Clang w/ sanitizers, sanitize: yes,
	compiler: clang-12, cxxstd: '03,11,14,17,20', os: ubuntu-20.04, stdlib: libc++, install: 'clang-12 libc++-12-dev libc++abi-12-dev' }

@pdimov Any preference?