craigcitro/r-travis

Tools for using R with Travis (http://travis-ci.org) in lieu of a website:

R+Travis

NATIVE R SUPPORT IN TRAVIS IS LIVE!

We've finally merged support based on this repo into Travis-CI, so you can simply use language: r to test your R packages!

• New users should check out the official Travis-CI R docs
• Existing users should check out the porting guide.

Setting up r-travis

This package has a simple shell script for use in running R package builds on travis, along with a sample .travis.yml file. One should be able to set up travis for their own project by:

• Copying sample.travis.yml to .travis.yml in your project.
  • Copying sample_revdeps.travis.yml to .travis.yml in your project will passively check reverse dependencies.
• Adding .travis.yml to your .Rbuildignore.
• Modifying .travis.yml to list any packages that must be installed from github (instead of CRAN).
• Turn on travis for your project. (Note that in some cases, it may take time for travis to start watching your repo.)
• Add a travis "build status" shield to your README file.

See the wiki for more extensive documentation and examples.

Linux or OS X builds

By default, builds are done on Linux. That is enabled by choosing language: c in the .travis.yml file. Builds on OS X can be enabled by choosing language: objective-c. See the comments in sample.travis.yml.

Currently, Travis-CI does not support builds that loop over platforms (such as Linux and OS X) in one build.

The future

My plan is to ultimately merge this into travis as a first-class citizen, so that the simplest config would simply say language: R. However, I'm using this repo as a staging ground to make sure I have the kinks worked out first. The end goal would be for the .travis.yml for an R project to be something as simple as

language: r
github_packages:
  - assertthat
  - devtools

r-travis's Issues

Build should throw error also in presence of WARNINGs

Warnings are like errors for the CRAN team.

grep WARNING "$package.Rcheck/00check.log"

Possible Windows builds

The Julia folks use a thing called WinRPM along with the openSUSE Build Service to provide Windows binaries.

Could we have Travis hook into this?

Add `travis-tool.sh` to `.Rbuildignore`

On failure, output complete check log

Because currently you only get a not very informative snippet from R CMD check.

Should we add --no-install-recommends ?

Looking somewhat more closely at a install preceding a test of a simple package, I see a metric ton of packages being pulled in. They shouldn't.

r-travis builds currently erroring?

e.g. https://travis-ci.org/hadley/dplyr/builds/13219926

Compile knitr/Sweave documents

I've started to upload copies of book chapters/publications to github. It occurred to me that it would rstraight forward to use r-travis to test knitr documents (without altering r-travis).

I've mocked up an example work-flow at https://github.com/csgillespie/travis-examples/tree/travis-knitr

The key ideas are:

• Create a DESCRIPTION file. This file only needs a Depends: section. The install_deps function then uses this file to install any packages that are needed by knitr/Sweave. In my example, the document just depends on knitr.
• In the .travis.yml file, turn latex on and tweak the install step.

Comments? Should I add this under the recipes section of the wiki?

@yihui any suggestions?

Use standard way of describing needed brew/debian packages

Maybe some fields in the DESCRIPTION ?

Create run-tests.sh?

Then the default script would just work. But maybe that's too magical.

Option to use R-nightly for testing

e.g. https://gist.github.com/trestletech/b5babf76487fa6c898b2

Use cran version of devtools

Now that 1.4 is available on cran.

Think about how to flag travis package downloads

So that eventually we could weed out of package download statistics.

Investigate RInstall failure

I've hit a failure that confuses me, likely because of the late hour:
https://travis-ci.org/craigcitro/r-travis/builds/13144905
that sure looks like it installed lubridate and then couldn't load it. i'm probably just confused.

Standardize 'verb' names

On the weekend, I wasted a build cycle or two because I had (mistakenly) assumed it was install_github when it is github_package.

Could we consider making every "install" operation be known as

  install_$foo

for different values of foo: aptget, github, cran, ... ie

  install_deps                   (already there)
  install_devtools              (new alias)
  install_aptget                 (new alias)
  install_r                         (new alias)
  install_r_binary              (new alias)
  install_github                 (new alias)

This would provide consistency as the other command also start with "action verbse" such as "dump" or "run".

Provide easy way to install binary versions of debian R packages

i.e. instead of install.packages("RCurl")), do apt-get install r-cran-rcurl

http://debian-r.debian.net/

Install additional Debian/Ubuntu packages

Can we add a hook to supply 'foo bar giz' so that 'sudo apt-get install foo bar giz' can be executed? My (trivial) test case is my tiny rhiredis package; it needs the hiredis C library from Debian/Ubuntu in order to build.

Need --no-manual too ?

First attempt to use this on Rcpp immediately bonked: https://travis-ci.org/RcppCore/Rcpp/builds/13562608

Looks like R CMD build wants --no-build-vignettes as well as --no-manual and the latter is currently missing.

DevtoolsInstall executed multiple times

devtools are installed upon each execution of GithubPackage or InstallDeps.

Install `qpdf` for checking pdf sizes

When I build my package I get the warning:

WARNING
‘qpdf’ is needed for checks on size reduction of PDFs

Make devtools optional

several folks chimed in on #17 asking to make devtools optional. i'm happy to do so if we need to; just to check, though, my understanding is that the two downsides of installing devtools are:

• install time (devtools has several dependencies)
• we use the bleeding edge devtools, which can sometimes bite.

one huge advantage for devtools is that it's able to read DESCRIPTION and install not just required packages, but also suggested and linkedto -- given that at least the former is almost always needed for tests to pass, it seems nice to make that happen without the user re-specifying dependencies in .travis.yml.

does that catch all the pros/cons? ( @eddelbuettel @yihui )

Provide command pkgurl_install to install pkg file from URL

Akin to aptget_install. Also, perhaps aptget_install should behave silently on OS X, and pkgurl_install behave silently on Linux, to allow "dual" configurations without extra clutter.

Split bootstrap into bootstrap_r and bootstrap_devtools ?

So that the logs look nicer - devtools has a lot of dependencies, some of which are pretty noisy (e.g. RCurl)

Error when lacking a tests directory

If my package doesn't contain a tests directory I get the following error:

cat: *.Rcheck/tests/*.Rout*: No such file or directory
The command "./travis-tool.sh run_tests" exited with 1.
Done. Your build exited with 1.

--no-build-vignettes should be optional

some packages only have HTML vignettes that do not require LaTeX, so it is safe to build vignettes, e.g. https://github.com/yihui/highr/blob/master/.travis.yml (this package has two vignettes: http://cran.r-project.org/web/packages/highr/)

devtools error

See http://stackoverflow.com/questions/20569485/ and it may be a PBKAC by the user. But I would like @krlmlr to take a look as I fear that some of his code may be complicating matters here. Shouldn't it default to devtools from CRAN now?

Update: The post is now deleted. Users with SO rep above a threshold can still see it. In his screenshot, r-cran-devtools appears.

Add function bootstrap_c2d4u (or similar) to add and update c2d4u apt repo

Helpful for #20 and #44.

Better use of notifications

It'd be nice to:

• get fewer emails
• only use some branches for the shield in README
• suggest not testing every branch by default, to make it easier on travis.

Use "$@" instead of "$*" and $*

Problem: "$*" expands to "$1 $2 $3...", i.e. one parameter. $* splits again, but may lead to a different value if the parameters contain spaces. "$@" is a robust alternative, even if we don't have spaces in our params.

Collect and output timing information

If you want faster tests, it's nice to know how long each step takes

open/close github issues on behalf of developper

Scenario:

I'm pushing some code to github that I think will fix a bug I'm tracking with github issues. I'd like the ability to embed the information that this push will maybe fix the bug. Something like:

git commit -m " #maybe_fixes #124"

We can imagine that e.g. using testthat, we can have the issue in the title of the test, e.g.

test_that( "this works #213", {
   ...
} )

Then, if the test passes and the issue is open, we close it.
And, if the test fails, but the issue is closed. we reopen it.

I'm pretty sure we can use github api for that (ping @cscheid). I guess this would need a user token.

Add routine to install LaTeX

Will be useful for preparing a cookbook, too.

install_deps doesn't fail if package installation fails

https://travis-ci.org/krlmlr/ProjectTemplate/builds/13674885

use options(warn=2)?

Multiple R packages in a single repo

Is it possible to have multiple packages in a single repository? For example, I have multiple (very small) R packages that I use for teaching.

I suppose it would be nice for each package to be validated, but I suppose a global validation would be fine.

How to specify username != hadley in github R package dependencies?

Hi,

(This looks like a really cool project. Thx for putting this together!)

How do I specify the github username in this line of the .travis.yml:

If you'd like to use the github version of any of your dependencies, one can simply add a line like

- ./travis-tool.sh install_github <package>

I looked at travis-tool.sh and it seems you do not handle this at all?
devtools then defaults to username = getOption("github.user") and in the end to "hadley".

And I get, e.g., this in Travis CI:

+ echo 'Installing github package: BBmisc'
Installing github package: BBmisc
+ Rscript -e 'library(devtools); library(methods); options(repos=c(CRAN="http://cran.rstudio.com")); install_github("BBmisc")'
Installing github repo BBmisc/master from hadley
Downloading BBmisc.zip from https://github.com/hadley/BBmisc/archive/master.zip
Error: client error: (406) Not Acceptable
Execution halted

Please note that I might also want to list dependencies that I do not develop but other people. So we just need a second argument right?

Best,

Bernd

Install non-R dependencies

One of my packages requires the R package rgdal, which depends on a non-R library called GDAL.

Does anyone else depend on rgdal? Perhaps there is a way to specify installing GDAL in the .travis.yml file?

uploading the check directory to dropbox

I played with this this morning. This directory has been uploaded to dropbox by travis. This is an intermediate measure but at least is a way to see build artefacts without dumping them in the log file.

This needs the dropbox.sh file from my fork of r-travis, inspired from https://github.com/andreafabrizi/Dropbox-Uploader by modified to fit nicely with travis.

It also needs some dropbox specific environment variables to handle authentification with dropbox etc ...

My dropbox branch of dplyr for example has :

env:
  global:
  - ACCESS_LEVEL=sandbox
  - secure: LELmW/uGnmc8gPcJcDMV5SeAy7eieqSrNTqW8mSrSShgu31ifQ9tzd+5U2SxL/LVit+oxT/oyXAKSjX8oebBC8qSZKMR33+kURxq8HHZ6eqhvJv3lh7wjvFpXdPdgr+HrtCqnKBqbMN0oQkMZAtBUTpIwf0K+Zw+eNon0GdYIc4=
  - secure: QOMc+iTz2UspKkm4J6XDNMxzALAzH8NCDR0J+s3JG2VWcHS0m0sbK6rIiTnztzonTP9052HfpgdUY8Culjpww7JamCQ3CYUA83NAdmXLrsSsUWC6emxkw0dCUi2exMVllNfy9uH+Dcj1ytZpDrabeBDS6CF4jbp7wpfMrfJeOO0=
  - secure: L3fKoY3SMpxsaLz/TUIy07j209E5ZCBFaXLEb3j0f+hYc97432xFDcBNjtE8TQ8X0GqnmCxZX/+45QRm2th7Hk2Ei1imIM2OMDQNnBFBnbzqN4qTrGkPCP6YEfKqw8NvxB6GkJEXgxE1zHIhTguicvDSPKNO1SUAirnzkRc1C6M=
  - secure: V94Mz3NGm5e39XPr776K3HMzX8m9Khz3DWwQgjE6gADn2kElm4ZgLYJ+hmTaW1W79vx/GDD6n3whPCP28oocOB/LLVX4zzY5tvjVegSo6gY1wRXPF+HEysjokxlMfVC68qS2qDr4f9FgJb0oREddJ5CxINZOL6suBYj9yzPhafA=

The 4 secure lines above are generated by travis encrypt ENV_VAR=value for the 4 environment variable that ./dropbox.sh needs:

APPKEY=xxx
APPSECRET=yyy
ACCESS_LEVEL=sandbox
OAUTH_ACCESS_TOKEN=xxx
OAUTH_ACCESS_TOKEN_SECRET=xxx

To get APPKEY and APPSECRET, we just create a dropbox app.
Then for the OAUTH_ACCESS_TOKEN and OAUTH_ACCESS_TOKEN_SECRET, I've used the original https://github.com/andreafabrizi/Dropbox-Uploader script. When you run it the first time and you follow through, it writes a ~/.dropbox_uploader file that contains the information.

This is still a bit involved but this is meant as a proof of concept of uploading whatever we create on travis into dropbox. I guess this could be applied to other things like google drive ...

With a bit of imagination, we could then have things like up to date vignettes, benchmark results.

Eventually I think it would make sense to have the information not necessarily on dropbox but on some service we would create as a community to make sense and present nicely.

wget | bash installation style

Copying travis-tool.sh, chmodding it and running bootstrap and bootstrap_devtools should happen in one command like

wget ... | bash

Can be done with Here documents and cat

Old travis-tool.sh remains a stub that notifies the user of the new syntax

Fragility from added repos

This is mostly an FYI and to see if we may want to make a few more things optional.

I just pulled and merged, and the build following the push failed as the PPA could not be reached: https://travis-ci.org/eddelbuettel/r-travis/builds/13442161

We need to keep an eye on events like this because if it happens too frequently we cannot rely on the extra .deb binaries.

Craig is of course free to close this as there is nothing operational in here...

Check silently fails to execute if source package is contained in the repo

https://github.com/craigcitro/r-travis/blob/master/scripts/travis-tool.sh#L221

https://travis-ci.org/krlmlr/ProjectTemplate/builds/13672566

Combine the best parts of r-travis with the best parts of R-travis

https://github.com/krlmlr/R-travis

InstallDeps needs to install Depends and LinkingTo packages

So my dplyr build doesn't fail :( - https://travis-ci.org/hadley/dplyr/builds/12956925

More control over testing

Depending on what you're working on, a full R CMD check might be overkill. It would be nice to have the ability to (e.g.) just run tests

Source travis-tool.sh instead of calling it

This is a suggestion for a rather radical change to the architecture of our tiny script. If necessary, it can be made fully backward-compatible.

Advantages:

1. We can export environment variables to the Travis build script (e.g., PATH on OS X, required for LaTeX)
2. We can control premature exit with success from the script to avoid testing build configurations we don't want to test
3. No need to prefix every call to our tool with ./travis-tool.sh -- this will be helpful for a later migration to travis-build, too
4. The travis-tool.sh script can be simplifed somewhat (of course not if backward compatibility is attempted)
5. Installation and bootstrapping can be carried out in one line, simplifying users' .travis.yml: source <(wget ...)

This works, because the entire build process in Travis CI is carried out in the same shell.

Configuration file?

Following up the discussion in #14.

Devtools install should use debian packages where available

esp for Rcurl.

Detect and use packrat

i.e. if packrat.lock found, run packrat::restore() instead of install_deps. See https://github.com/rstudio/packrat

(More of a long-term idea than something immediate)

Cache package install

Yet another idea for reducing package install time: http://about.travis-ci.org/docs/user/caching/ discusses the standard travis for caching language specific dependencies between builds. It may be as simple as:

cache:
  bundler: true
  directories:
  - library

assuming that we install all packages in library

install_deps should look into other repositories (Bioconductor, Omegahat, ...) by default

This is now really easy:

setRepositories(ind=1:8)

Would there be a reason not to?

Use sudo only where necessary

Better way of setting CRAN url

Should set it in appropriate .Rprofile (not sure how to set that up though)