lionsad / drupal_ti Goto Github PK
View Code? Open in Web Editor NEWDrupal - Travis Integration
Drupal - Travis Integration
Thanks a lot for publishing this code! The more I read through it, the more useful features I find.
https://www.drupal.org/node/2489956 suggests
$ cd core
$ export SIMPLETEST_DB=mysql://root:@localhost/dev_d8; ./vendor/bin/phpunit --testsuite=kernel
I am currently using phpunit-core with SIMPLETEST_DB set in .travis.yml like this:
script:
- drupal-ti --include .travis-phpcs.sh
- export SIMPLETEST_DB=mysql://root:@127.0.0.1/drupal_travis_db; drupal-ti script
which, honestly, I don't really like but it works for now . :)
I was thinking about two options:
Which approach is correct?
drafty specifies field_collection and entity_translation only as test_dependencies[], which drush does not support.
It also fails on drush/drush:master completely as it specifies drafty_enforce, but drush can't find it - even though this is in drafty.
I am running all drupal core simpletests with my module enabled, and I was really confused to see travis failing my build even though all the tests were passing. It turns out that the regex is catching some of the drupal test descriptions:
egrep -i "([1-9]+ fail)|(Fatal error)|([1-9]+ exception)" /tmp/simpletest-result.txt && exit 1
UserLoginTest.php Drupal\user\Tests\UserLoginTest
Raw "There have been more than 3 failed login attempts for this account."
"[1-9]+ fail" catches "3 failed"
I don't know if this is due to an recent changes or the release of Drupal 8, but it appears that 'drush dl' commands are working inconsistently between different PHP versions. The obvious reason for that of course is the version of Drush being installed on 5.3 is different to those above.
The issue specifically is when I need to add a test dependency via Drush for a Drupal 7 module, 'drush dl' appears to be getting the Drush 8 release instead of the Drupal 7.
On this, it does look like it's an indicator that the 'drush use' stuff that I implemented previously isn't entirely working correctly, so I will have a look into that as soon as possible.
See the following test log to see the issue in practice: https://travis-ci.org/Decipher/file_aliases/jobs/93308484#L773
A simple workaround is to specify the version of the module in the 'drush dl' command: https://travis-ci.org/Decipher/file_aliases/jobs/93419634#L247
Issue #2551981: Add --directory option to run-tests.sh test discovery.
This is not in Drupal core, but it could potentially cause issues. Just pinging you here so you're aware of the issue.
This did not make it into the behat branch:
Having an example of how to use https://github.com/squizlabs/PHP_CodeSniffer or https://github.com/FriendsOfPHP/PHP-CS-Fixer would be nice for module:
Here's what we had in our .travis.yml file for a d8 module before switching to drupal_ti:
env:
global:
PHPCS_VERSION='2.0.*@dev'
CODER_VERSION='8.2.0-alpha2'
before_install:
composer global require squizlabs/php_codesniffer:$PHPCS_VERSION
composer global require drupal/coder:$CODER_VERSION
ln -s ~/.composer/vendor/drupal/coder/coder_sniffer/Drupal ~/.composer/vendor/squizlabs/php_codesniffer/CodeSniffer/Standards/
script:
phpcs --report=full --standard=Drupal $TRAVIS_BUILD_DIR/drupal/modules/$MODULE_NAME
Pointing you towards the solution we used to fix the rate limitation failures:
https://www.drupal.org/node/2581389#comment-10492734
In drupal-7.sh
and drupal-8.sh
there is a command which run drush in the following way:
~/.composer/vendor/bin/drush.php
so I've the following error:
php -d sendmail_path=/bin/true /home/travis/.composer/vendor/bin/drush.php --yes site-install testing --db-url=mysql://root:@127.0.0.1/drupal
Could not open input file: /home/travis/.composer/vendor/bin/drush.php
Since I'd like to have my drush
available globally as I'm using drupal_ti
on top of my other CI tests, I've specified the following environment variable: COMPOSER_BIN_DIR=~/bin
which automatically link drush
binary into ~/bin
which is available by default in Travis CI by just running drush
(so I don't need to modify my PATH
). This way is also described in drush
docs.
So I wondering if there is a better way of running drush
. It seems this syntax is only used in order to handle mails (php -d sendmail_path=$(which true)
).
In my .travis.yml
, my approach is a bit different (by installing mailcatcher
, so there can be some tests related to received e-mails):
- gem install mailcatcher && mailcatcher -v
- echo 'sendmail_path="/usr/bin/env catchmail"' | tee -a "$(php --ini | grep "Loaded Configuration" | awk '{print $4}')"
In this case, we can install site in normal way, which is: drush --yes site-install
, so we're not depending on any hardcoded paths and mails are handled by catchmail
.
https://travis-ci.org/mglaman/commerce_demo/jobs/201552451
https://travis-ci.org/drupalcommerce/commerce/jobs/201116282
There's a weird issue.
0.65s$ drupal-ti script
Cannot open file "/home/travis/build/drupalcommerce/drupal-8/drupal/modules/commerce/.php".
0.47s$ drupal-ti script
Cannot open file "/home/travis/build/mglaman/drupal-8/drupal/modules/commerce/demo/.php".
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit-core/script.sh exited with a failure.
My module requires a library (in vendor) that is installed from the project composer in this way:
{
"repositories": [
{
"type": "package",
"package": {
"name": "phpoffice/phpspreadsheet",
"version": "0.0",
"dist": {
"url": "https://github.com/PHPOffice/PhpSpreadsheet/archive/develop.zip",
"type": "zip"
}
}
}
],
"require": {
"phpoffice/phpspreadsheet": "~0.0"
},
"autoload": {
"psr-4": {
"PhpOffice\\PhpSpreadsheet\\": "vendor/phpoffice/phpspreadsheet/src/PhpSpreadsheet"
}
}
}
I guess is easy to download and place the code in vendor/phpoffice/phpspreadsheet, but how to fix the autoload?
I'm having some trouble getting Behat to run in a contrib module for D8: stevector/template_mapper#2 My last couple of commits have been switching the api driver between drush
and drupal
. I'm getting different errors with each.
Do you know of any contrib modules using drupal_ti that I could use as a comparison? I've looked at Panopoly but it seems like a much more advanced use case.
.travis.yml.dist
defines DRUPAL_TI_MODULE_NAME
, which describes the project name, but not any specific module, since a project can contain multiple modules. In addition, DRUPAL_TI_SIMPLETEST_GROUP
can probably contain multiple group names, but that's not really clear.
Is there any way to get drupal_ti working together with BrowserTestBase?
The failure in https://travis-ci.org/d8-contrib-modules/file_encrypt/jobs/135598949 seems to cause issues about it
Currently, the latest version of Selenium is 2.48.2. However, if I set DRUPAL_TI_BEHAT_SELENIUM
to that value, I get:
--2015-10-13 13:02:17-- http://selenium-release.storage.googleapis.com/2.48.2/selenium-server-standalone-2.48.2.0.jar
Resolving selenium-release.storage.googleapis.com (selenium-release.storage.googleapis.com)... 173.194.206.128, 2607:f8b0:400d:c06::80
Connecting to selenium-release.storage.googleapis.com (selenium-release.storage.googleapis.com)|173.194.206.128|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2015-10-13 13:02:17 ERROR 404: Not Found.
The correct URL is actually:
http://selenium-release.storage.googleapis.com/2.48/selenium-server-standalone-2.48.2.jar
So, it needs to use the first 2-parts of the version number for the directory bit of the URL, but then the full 3-parts for the filename part.
I am using travis to test changes to my Drupal module before merging them back into 8.x-1.x.
However I think that since environments/drupal-8.sh drupal_ti_ensure_module_linked() now gets the module from composer using '*@dev', rather than using the commit of the module that I am actually testing, it always gets a release version (eg alpha-1) from drupal.org.
Why is this better than using drupal_ti_ensure_module_linked() as defined in functions/drupal.sh which uses the commit checked out by travis?
When a module has a dependency on composer_manager, the build fails. It is basically stuck at this point:
The following projects have unmet dependencies: [ok]
url_embed requires composer_manager
Would you like to download them? (y/n): y
Install location /home/travis/build/drupal-media/drupal-8/drupal//modules/composer_manager already exists. Do you want to overwrite it? (y/n):
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
Link to the relevant build in case you want to check: https://travis-ci.org/drupal-media/url_embed/jobs/70604609#L208
DRUPAL_TI_DB is required even though already present in DRUPAL_TI_DB_URL.
In addition to that travis can create a DB itself, too, now.
I've got the following error in this build:
$ drupal-ti before_script
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/simpletest/before_script.sh exited with a failure.
My .travis.yml file.
Installed via: composer global require -n --prefer-source lionsad/drupal_ti:1.*
I have a Drupal module that relies on Composer Manager module and the Composer extension for Drush.
Everything goes well till after the Composer extension is installed for Drush. I need to find how I can inject a clear cache on drush after the extension loads.
Here is the travis log output
Download and install the Drush Composer extension? (y/n): y
Project composer (8.x-1.x-dev) downloaded to [success]
/home/travis/.drush/composer.
Project composer contains 0 modules: .
The drush command 'composer' could not be found. [error]
drupal-media/url_embed#6 should not be needed for each project.
Support properly via an option within DRUPAL_TI:
# Auto-detect if composer.json with type: 'drupal-module' is present
# Use 0 to disable, use 1 to enable.
DRUPAL_TI_USE_COMPOSER_MANAGER="auto"
Needs to be supported by simpletest and behat runners.
Somewhat related to #25, currently the installation profile is hardcoded, but it would be nice if we would set it in the .travis.yml file.
Relatively simple issue, PR coming when I get the time to write it.
1.4.2 was released some time ago, and master contains some fixes that are necessary because of newer Drush releases. @LionsAd asked me to ping him in this issue.
Hello. I'm having a problem running the test. You can see the error and the entire output at
https://travis-ci.org/Tantacom/tantacom-web/jobs/134472696#L631
and my travis.yml file at https://github.com/Tantacom/tantacom-web/pull/5/files#diff-354f30a63fb0907d4ad57269548329e3R24
It looks like drupal is somehow installed two times... one under /home/travis/build/Tantacom/tantacom-web/
and the other /home/travis/build/Tantacom/drupal-8/drupal/
any ideas of what could i be doing wrong?
Commit:
drupalcommerce/commerce@bec25f7
Result:
/home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh: line 13: ./vendor/bin/phpunit: > No such file or directory
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh exited with a failure.
https://travis-ci.org/commerceguys/commerce/jobs/76440492
Probably needs a similar fix as mradcliffe@729b806
http://docs.travis-ci.com/user/migrating-from-legacy/
What are the restrictions? #
Using sudo isn’t possible (right now) #
Our new container infrastructure uses Docker under the hood. This has a lot of benefits like faster boot times and better utilization of resources. But it also comes with some restrictions. At this point, it’s not possible to use any command requiring sudo in your builds.
If you require sudo, for instance to install Ubuntu packages, a workaround is to use precompiled binaries, uploading them to S3 and downloading them as part of your build,then installing them into a non-root directory.
Databases don’t run off a memory disk #
On our legacy infrastructure, both MySQL and PostgreSQL run off a memory disk to increase transaction and query speed. This can impact projects making heavy use of transactions or fixtures.
Inside before_script
:
before_script:
- drupal-ti before_script
- drupal-ti --include drupal_ti/before/before_script.sh
- drupal-ti --include drupal_ti/before/runners/behat/before_script.sh
And I have a file like this drupal_ti/before/runners/behat/before_script.sh
:
#!/bin/bash
# Add an optional statement to see that this is running in Travis CI.
echo "running drupal_ti/before/runners/behat/before_script.sh"
set -e $DRUPAL_TI_DEBUG
export BEHAT_PARAMS='{"extensions":{"Behat\\MinkExtension":{"base_url":"DRUPAL_TI_WEBSERVER_URL:DRUPAL_TI_WEBSERVER_PORT/"},"Drupal\\DrupalExtension":{"drush":{"root":"DRUPAL_TI_DRUPAL_DIR"},"drupal":{"drupal_root":"DRUPAL_TI_DRUPAL_DIR"}}}}'
BEHAT_PARAMS=`echo $BEHAT_PARAMS | sed -e "s#DRUPAL_TI_WEBSERVER_URL#$DRUPAL_TI_WEBSERVER_URL#" -e "s#DRUPAL_TI_WEBSERVER_PORT#$DRUPAL_TI_WEBSERVER_PORT#" -e "s#DRUPAL_TI_DRUPAL_DIR#$DRUPAL_TI_DRUPAL_DIR#" -e "s#DRUPAL_TI_DRUPAL_DIR#$DRUPAL_TI_DRUPAL_DIR#"`
export BEHAT_PARAMS
echo $BEHAT_PARAMS
But it seems the BEHAT_PARAMS
is not considered when the tests are run. And it fails with error:
[InvalidArgumentException]
Driver "drupal" is not registered.
Any idea why this is happening?
As a workaround I am setting BEHAT_PARAMS
under before_script
like this:
# Setting Behat environment
- DRUPAL_DIR="$TRAVIS_BUILD_DIR/../$DRUPAL_TI_ENVIRONMENT/drupal"
- BEHAT_PARAMS='{"extensions":{"Behat\\MinkExtension":{"base_url":"DRUPAL_TI_WEBSERVER_URL:DRUPAL_TI_WEBSERVER_PORT/"},"Drupal\\DrupalExtension":{"drush":{"root":"DRUPAL_DIR"},"drupal":{"drupal_root":"DRUPAL_DIR"}}}}'
- BEHAT_PARAMS=`echo $BEHAT_PARAMS | sed -e s#DRUPAL_TI_WEBSERVER_URL#$DRUPAL_TI_WEBSERVER_URL# -e s#DRUPAL_TI_WEBSERVER_PORT#$DRUPAL_TI_WEBSERVER_PORT# -e s#DRUPAL_DIR#$DRUPAL_DIR# -e s#DRUPAL_DIR#$DRUPAL_DIR#`
- export BEHAT_PARAMS
Seeing this in commerce tests:
https://travis-ci.org/joshuataylor/commerce/jobs/81239645
/home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh: line 13: ./vendor/bin/phpunit: No such file or directory
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/phpunit/script.sh exited with a failure.
Document new parameter from PR
Currently, "before_script" performs the following actions:
Most 8.x module dependencies do not have releases on drupal.org yet. As such, step 3 fails because module dependencies are not found. There have also been known Drush issues with downloading and discovering dependencies (which is out of your hands).
I would develop drupal-ti before_script
to call drupal-ti site-install
(steps 1 and 2 noted above) and drupal-ti module_install
(steps 3 above).
This enables a user to call drupal-ti before_script
if they have no issues with dependencies, or they can call drupal-ti site-install
, their own logic, and then drupal-ti module_install
if they need more customization.
I've been playing with Drupal Ti for a few hours now, and have mostly got to where I want to be, however I am having one issue that I can't really seem to figure out.
My Behat tests using DrupalExtention seem to be able to communicate with the Drupal API. On my local tests it runs fine, but on Travis CI I'm not getting any luck.
Specifically, what I'm doing in Behat is running module_enable() in a @BeforeSuite hook to turn on a test module, but I'm getting the following error on Travis:
PHP Fatal error: Call to undefined function module_enable() in /home/travis/build/Decipher/test/tests/behat/features/bootstrap/WysiwygFieldsFeatureContext.php on line 14
https://travis-ci.org/Decipher/test/jobs/73371096#L443
You can see my test repo (and it's excessive test commits) at https://github.com/decipher/test
It is getting crowded for the environment variables. Ensure that things like
DRUPAL_TI_BEHAT_SCREENSIZE_COLOR 1280x1024x16
etc.
can be specified by runners/behat like defaults/behat.sh:
drupal_ti_default_configuration DRUPAL_TI_BEHAT_SCREENSIZE_COLOR "1280x1024x16"
and commented out in .travis.yml, so that defaults can be specified easily by the runners.
I've just committed a change to composer_manager that makes it use the root vendor directory instead of the core one: https://www.drupal.org/node/2452781#comment-10240975
Having both vendor/ and core/vendor creates a weird situation for phpunit.
I can run phpunit from the root using
./vendor/bin/phpunit -c core
But I can no longer run phpunit from core/vendor/bin/phpunit, because it ends up including both autoloaders somehow, and crashes with:
PHP Fatal error: Cannot redeclare composerRequireDrupal8() (previously declared in >/home/travis/build/commerceguys/drupal-8/drupal/core/vendor/composer/autoload_real.php:56) in >/home/travis/build/commerceguys/drupal-8/drupal/vendor/composer/autoload_real.php on line 55
This is problematic because simpletest hardcodes the core phpunit, making the build fail.
Not yet sure what the proper fix is, wanted to report it first.
Hi Fabian,
Hopefully you can help me out with something. In epically large amount of test commits based repo, http://github.com/Decipher/test, I'm trying to get some behat tests for my Wysiwyg Fields module sorted out.
The tests work on my local machine with both selenium/firefox and with phantomjs, but no matter what I do, I can't get them to work on Travis.
The major complexity is that they are CKEditor dialog based tests, which requires timing. I have take two approaches, both which work locally, one using the DrupalExtension 'wait for AJAX' step, and the current with a custom 'wait till :element visible' approach. The first approach just failed outright on Travis, whereas the current just sits there until I kill the test (as it's a loop with no timeout).
If there's anything you can think of I would greatly appreciate it.
Hi, this might be me not entirely understanding the travis test stack properly, but I have a request.
I would like to be able to run all drupal simpletests with my module enabled when I push code to a repo. This is so as to make sure that my module does not break core functionality.
As this takes hours however, I would like to only run tests that my module provides when creating pull requests, so that we don't have to spend hours waiting for tests before merging code.
Drupal_ti uses the DRUPAL_TI_SIMPLETEST_GROUP variable to tell the simpletest runner which tests to limit by, and TRAVIS_PULL_REQUEST tells me whether on not I want the runner to filter tests.
I've attempted to change the variable like this:
if [ "${TRAVIS_PULL_REQUEST}" = "false" ]; then
echo 'Scheduling all simpletests to be run.'
else
echo 'Scheduling my_module simpletests to be run.'
export DRUPAL_TI_SIMPLETEST_GROUP='my_module'
fi
But this doesn't seem to work as by the time the simpltest runner 'script.sh' runs, it has reset DRUPAL_TI_SIMPLETEST_GROUP to whatever is set in the .travis.yml.
Is there any way that I can achieve this without patching the runner script as part of my environment setup?
Travis build : https://travis-ci.org/naveenvalecha/autologout/jobs/110087030
Travis yml file : https://github.com/naveenvalecha/autologout/blob/8.x-1.x/.travis.yml
It'd be really nice if a Drush alias was created and 'drush use'd so that interactions with the Drupal site could be done without the need of navigating the filesystem.
Hello
I'm using drupal_ti to test my Drupal module (https://github.com/Ozmodiar/DIC) using PHPUnit and Simpletest at the same time but when following the installation instructions, something went wrong.
$ composer global require "lionsad/drupal_ti:1.*"
Changed current directory to /home/travis/.composer
./composer.json has been created
Package "lionsad/drupal_ti" listed for update is not installed. Ignoring.
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Generating autoload files
$ drupal-ti before_install
/home/travis/build.sh: line 41: drupal-ti: command not found
The command "drupal-ti before_install" failed and exited with 127 during .
Your build has been stopped.
My temporary solution is to add a few extra composer (global) updates
, such as:
before_install:
- composer self-update
- composer global require "lionsad/drupal_ti:1.*"
- composer global update
- drupal-ti before_install
install:
- drupal-ti install
before_script:
- composer global update
- composer update --dev
- drupal-ti before_script
Is there a better solution to do this?
Running phpunit tests should not require a Drupal installation as there should be no database state for those tests. It should only need to download Drupal.
Solution should be to decouple download and install tasks from the ensure drupal and ensure module functions.
Since a couple of days my build is errored each times, it seems that it's a requirement issue from drush :
Changed current directory to /home/travis/.composer
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Problem 1
- Installation request for drush/drush dev-master -> satisfiable by drush/drush[dev-master].
- drush/drush dev-master requires codegyre/robo ~1.0.0-beta1 -> satisfiable by codegyre/robo[1.0.0-beta1] but these conflict with your requirements or minimum-stability.
Installation failed, reverting ./composer.json to its original content.
Error: /home/travis/.composer/vendor/lionsad/drupal_ti/runners/simpletest/install.sh exited with a failure.
The command "drupal-ti install" failed and exited with 1 during .
Your build has been stopped.
Don't have time to dig now, will do tomorrow.
full stack is visible here : https://travis-ci.org/mespronos/mespronos/jobs/129421306
Unlike unit tests, kernel tests are added to the right test group.
That means they're run twice, once with phpunit and once with run-tests.sh
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.