Package-Builder
This repository contains build and utility scripts used for continuous integration builds on the Travis CI environment. It offers many extension points for customizing builds and tests.
Build prerequisites
Package-Builder is intended to be used as part of Travis CI tests, and will operate on both Ubuntu 14.04 and macOS. At a minimum, the .travis.yml file of your application will look something like this:
$ cat .travis.yml
matrix:
include:
- os: linux
dist: trusty
sudo: required
- os: osx
osx_image: xcode9
sudo: required
before_install:
- git clone https://github.com/Kitura/Package-Builder.git
script:
- ./Package-Builder/build-package.sh -projectDir $TRAVIS_BUILD_DIR
If you need to install system-level dependencies such as libmysqlclient-dev, you can do so in the before_install section of the .travis.yml file so that the Travis CI build environment is ready for compilation and testing of your Swift package.
How to start the build-package.sh script
This script must be started form the folder that contains your Swift package. Also, please note that the projectDir argument passed to the script should be the directory of the whole repository. For most projects, this is the same as the folder that contains your Swift package, as shown in the example above. However, there are repositories where the Swift packaage is a sub-folder in the main project.
Providing custom credentials
It is not uncommon for swift packages to need to connect to secure services, offerings, and middleware such as databases. To do this, credentials are needed from properties files. To ensure the security of these credentials, many teams use private repositories to store these credentials while their public ones contain dummy files like the one below:
$ cat configuration.json
{
...
"credentials": {
"url": "<url>",
"name": "<name>",
"password": "<password>"
}
...
}
The true credentials, show below, should be stored in a private repository:
$ cat configuration.json
{
...
"credentials": {
"url": "api.ng.bluemix.net/v2/authenticate",
"name": "[email protected]",
"password": "passw0rd"
}
...
}
In order to meet this need, Package-Builder will copy and overwrite these dummy files with the credentials from the private repository. To leverage this functionality, be sure to clone the credentials in the before_install section, and then use the following in your .travis.yml, pointing towards the folder where the cloned credentials exist:
script:
- ./Package-Builder/build-package.sh -projectDir $TRAVIS_BUILD_DIR -credentialsDir <path to credentials>
Codecov
Codecov is used in Package-Builder to determine how much test coverage exists in your code. Codecov allows us to determine which methods and statements in our code are not currently covered by the automated test cases included in the project. Codecov performs its analysis by generating an Xcode project.
To turn on Codecov, you need to add the following to your .travis.yml file:
env:
global:
- CODECOV_ELIGIBLE=trueFor example, see the current test coverage for the Swift-cfenv package.
Please note that Codecov is only leveraged when executing builds on the macOS platform.
SonarCloud
In parallel to Codecov, you can also use SonarCloud. SonarCloud will provide you both with code coverage and code analysis tools.
Just as Codecov, SonarCloud is only leveraged when executing builds on the macOS platform.
A little customization to your .travis.yml file is needed:
env:
global:
- SONARCLOUD_ELIGIBLE=true
- SONAR_LOGIN_TOKEN={your login token -- better add this in Travis directly}Feel free to read the source for more info: sonarcloud.sh
It is recommended to use a sonar-project.properties at the root of your project. Here's a good example: sonar-project.properties. The whole Swift-Travis-Sonarcloud-CI repo is a good starting point for both Travis-CI & Sonarcloud.
You can also provide a .swift-sonarcloud file with your custom sonar-scanner command.
Auto Jazzy Docs Build
Jazzy provides automatic documentation construction. To simplify the process of updating public facing api/documentation, package builder can automate the creation and pushing of updated docs for a Pull Request.
To indicate that documentation should be generated, add the jazzy-doc label to the Pull Request.
In order for a PR to receive automatic documentation generation, the following must be configured:
- The Travis configuration for the repository must define the following environment variables, specifying the credentials of a user that has sufficient permissions to push to PR branches:
GITHUB_USERNAMEGITHUB_PASSWORDGITHUB_EMAIL
- The repository must have a
jazzy-doclabel defined - The
.travis.yamlfor the project must contain one macOS build withenv: JAZZY_ELIGIBLE=true - The PR must have the
jazzy-doclabel applied
Once the regular build has executed, Jazzy will be run for MacOS builds and the resulting documentation pushed to the PR branch in a new [jazzy-doc] commit. Docs will be generated for each new commit to the PR branch whose commit message does not contain the text [jazzy-doc].
Custom Xcode project generation
If for Codecov, you need a custom command to generate the Xcode project for your Swift package, you should include a .swift-xcodeproj file that contains your custom swift package generate-xcodeproj command.
Custom code coverage
If you need to run a custom command to generate code coverage for your Swift package, you should include a .swift-codecov file that contains your command.
Custom SwiftLint
SwiftLint is a tool to enforce Swift style and conventions. Ensure that your team's coding standard conventions are being met by providing your own .swiftlint.yml in the root directory with the specified rules to be run by Package-Builder. For now each project should provide their own .swiftlint.yml file to adhere to your preferences. A default may be used in the future, but as of now no SwiftLint operations are performed unless a .swiftlint.yml file exists.
Please note that SwiftLint is only leveraged when executing builds on the macOS platform.
Using different Swift versions and snapshots
Package-Builder uses, by default, the most recent release version of Swift, which at the time of writing is 4.0.3. If you need a specific version of Swift to build and compile your repo, you should specify that version in a .swift-version file in the root level of your repository. Valid contents of this file include release and development snapshots from Swift.org.
$ cat .swift-version
swift-DEVELOPMENT-SNAPSHOT-2017-02-14-a
Testing with multiple Swift versions
To test your package using a different version of Swift than the one specified in your .swift-version file, simply add the SWIFT_SNAPSHOT environment variable to your .travis.yml file in each one of the entries under the matrix section as shown below:
$ cat .travis.yml
matrix:
include:
- os: linux
dist: trusty
sudo: required
- os: linux
dist: trusty
sudo: required
env: SWIFT_SNAPSHOT=3.1.1
before_install:
- git clone https://github.com/Kitura/Package-Builder.git
script:
- ./Package-Builder/build-package.sh -projectDir $TRAVIS_BUILD_DIR
In this example above, the first build uses the version specified in the .swift-version of the project, or the default version supported by Package-Builder. The second one declares a SWIFT_SNAPSHOT environment variable, which overrides the default and .swift-version versions for that build.
Testing under Docker
To test your package using a different version of Linux, add the DOCKER_IMAGE environment variable to your .travis.yml file in each one of the entries under the matrix section as shown below:
$ cat .travis.yml
matrix:
include:
- os: linux
dist: trusty
sudo: required
env: SWIFT_SNAPSHOT=4.1.3
- os: linux
dist: trusty
sudo: required
env: DOCKER_IMAGE=ubuntu:16.04 SWIFT_SNAPSHOT=4.1.3
before_install:
- git clone https://github.com/Kitura/Package-Builder.git
script:
- ./Package-Builder/build-package.sh -projectDir $TRAVIS_BUILD_DIR
In the above example, the first build uses Ubuntu 14.04 (Trusty) which is supported natively by Travis. The second build uses Trusty to download a 16.04 (Xenial) Docker container, and will then re-execute the Package-Builder command within that container.
Additional environment variables
Selected environment variables are passed through to the container. These are currently: SWIFT_SNAPSHOT and KITURA_NIO. Additional environment variables can be passed through by setting the DOCKER_ENVIRONMENT variable as follows:
env: DOCKER_IMAGE=ubuntu:16.04 DOCKER_ENVIRONMENT="CUSTOMENV1 CUSTOMENV2"
Additional system packages
A number of system packages are installed within the Docker container by default (this includes pkg-config for SwiftPM, and packages required by Package-Builder itself). Additional system package dependencies can be specified by setting the DOCKER_PACKAGES variable as follows:
env: DOCKER_IMAGE=ubuntu:16.04 DOCKER_PACKAGES="libSomePackage someOtherPackage"
Custom build and test commands
If you need a custom command for compiling your Swift package, you should include a .swift-build-linux or .swift-build-macOS file in the root level of your repository and specify in it the exact compilation command for the corresponding platform.
$ cat .swift-build-linux
swift build -Xcc -I/usr/include/postgresql
If you need a custom command for testing your Swift package, you should include a .swift-test-linux or .swift-test-macOS file in the root level of your repository and specify in it the exact testing command for the corresponding platform.
$ cat .swift-test-linux
swift test -Xcc -I/usr/include/postgresql
If you require more granularity than the platform files above provide you can also set the CUSTOM_BUILD_SCRIPT and CUSTOM_TEST_SCRIPT environment variables in your travis configuration. The scripts these environment variables point to will be executed in place of the platform custom scripts or default commands.
$ cat .build-ubuntu1404
swift build -Xlinker -L/usr/lib -Xcc -I/usr/include/ -Xcc -I/usr/include/mysql/
$ cat .test-ubuntu1404
swift test -Xlinker -L/usr/lib -Xcc -I/usr/include/ -Xcc -I/usr/include/mysql/
$ cat .travis.yml
matrix:
include:
- os: linux
dist: trusty
services: docker
env:
- DOCKER_IMAGE=ubuntu:14.04 CUSTOM_BUILD_SCRIPT=.build-ubuntu1404 CUSTOM_TEST_SCRIPT=.test-ubuntu1404
sudo: required
Custom swift test arguments
If you only need to provide arguments to the swift test command, rather than providing a customized test script, you can define the SWIFT_TEST_ARGS environment variable. For example:
SWIFT_TEST_ARGS="--parallel --num-workers=16"
Custom configuration for executing tests
Sometimes, a dependency must be set up before the testing process can begin. You may also have the need to execute certain actions after your tests have completed (e.g. shutting down a server). Package-Builder provides an extension point to do this; you can include a before_tests.sh and/or a after_tests.sh file containing the commands to be executed before and after the tests.
These files should be placed in a folder structure that matches the outline shown below (see the linux, osx, and common folders):
Before Tests: The linux/before_tests.sh and osx/before_tests.sh scripts will be executed first if present, followed by common/before_tests.sh. Once complete, the tests will commence.
After Tests: After the tests are performed, common/after_tests.sh is executed first, followed by linux/after_tests.sh or osx/after_tests.sh.
Troubleshooting
If there is a crash during the execution of test cases, Package-Builder will perform a log dump to provide meaningful diagnosis of where the failure has occurred.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent