Giter Site home page Giter Site logo

lullabot / drupal9ci Goto Github PK

View Code? Open in Web Editor NEW
161.0 24.0 55.0 2.54 MB

Command Line Interface for implementing Continuous Integration in Drupal 9

License: GNU General Public License v3.0

Shell 63.62% Dockerfile 9.24% Go 27.14%
circleci workflow continuous-integration drupal composer-drupal travis-ci gitlab-ci github-actions ci coverage

drupal9ci's Introduction

Drupal 9 CI

Test Create release Publish Docker image codecov

This repository provides the foundation to implement Continuous Integration in a Drupal 9 project using any of the following CI providers:

If you want to test an individual module instead of a Drupal project, see Andrew Berry's drupal_tests.

Here is a clip that shows how it works for CircleCI.

Requirements

The scripts assume that you have a Drupal 9 project created using drupal-project, which sets a well known foundation. If your project's directory structure differs from what drupal-project sets up, you will need to adjust the CI scripts.

It's also recommended to adjust your project to add a subset of the settings.php file into version control and rely on settings.local.php for setting the database connection. The CI scripts have their own settings.local.php which is copied at build time into web/sites/default.

DocumentRoot: web vs docroot

By default, Apache and the rest of the code is set up to have the DocumentRoot folder as web. If your project uses docroot instead, the easiest thing to do is to symlink one to the other. This can be done adding a line like this one before running Apache (you will need to adapt paths).

Alternatively you can go through the files and replace as needed. You can see a list of affected files in this comment.

Installation

Demo repository | Deep dive article

Download the drupal9ci binary from https://github.com/Lullabot/drupal9ci/releases into the project root and run the command as ./drupal9ci and select CircleCI. Once complete, continue below to complete the setup.

Sign up at CircleCI and allow access to your project's repository.

CircleCI watch

Happy CI-ing! :-D. From now every time you create a pull request, CircleCI will run the set of jobs and report their result like in the following screenshot:

CircleCI pull request

For an overview of the CircleCI features, have a look at this article from the Lullabot blog.

Using a custom Docker image

The CircleCI configuration file uses a custom Docker image that extends from the official Drupal image. If this image does not fit your project's architecture then consider creating your own image based out of it.

Demo repository | Deep dive article

Download the drupal9ci binary from https://github.com/Lullabot/drupal9ci/releases into the project root and run the command as ./drupal9ci and select Travis CI. Once complete, continue below to complete the setup.

Sign up at Travis CI and allow access to your project's repository:

Travis watch

Happy CI-ing! :-D. From now on every pull request will trigger a build in Travis and its progress will be visible like in the following screenshot:

Travis pull request

For you to see the result of the individual jobs, you need to click at the Details link from the above screenshot:

Travis CI jobs

Setting up code coverage reports

Coveralls.io is a third party tool that can host and present PHPUnit code coverage reports in a neat way within a pull request. Here is how to set it up:

Register at https://coveralls.io using your GitHub account and then add your repository like in the following screenshot:

Coveralls add repository

Then take the chance to adjust a couple settings to get cleaner feedback in pull requests:

Coveralls settings

That's it! Here is a sample report which you can see by clicking at Details at the pull request's status message:

Coveralls report

Demo repository | Deep dive article

Download the drupal9ci binary from https://github.com/Lullabot/drupal9ci/releases into the project root and run the command as ./drupal9ci and select GitLab CI. Once complete, continue below to complete the setup.

Review, commit, and push the resulting changes. After doing that, navigate to the project's homepage at GitLab and open the CI / CD >> Pipelines section. You should see a running pipeline like the following one:

GitLab pipeline

Demo repository

Download the drupal9ci binary from https://github.com/Lullabot/drupal9ci/releases into the project root and run the command as ./drupal9ci and select GitHub Actions. Once complete, continue below to complete the setup.

Review, commit, and push the resulting changes. After doing that, navigate to the repository's homepage at GitHub and open the Actions tab. You should see a running workflow like the following one:

Github actions

Download the drupal9ci binary from https://github.com/Lullabot/drupal9ci/releases into the project root and run the command as ./drupal9ci and select Bitbucket. Once complete, continue below to complete the setup.

Review, commit, and push the resulting changes. After doing that, navigate to the repository's homepage at Bitbucket and open the Pipelines tab. You should see a running workflow like the following one:

Bitbucket pipelines

Setting up the Behat and Cypress jobs for all platforms

The Behat and Cypress jobs require a running Drupal 9 site. The repository contains the code, but for running tests in a realistic environment you need:

1. A recent copy of the production environment's database

There are several ways to accomplish this:

Using a prepopulated Docker database image

See Achieve Rocketship-Fast Jobs in CircleCI by Preinstalling the Database

Travis

If you have Drush site aliases, and your repository is private, then follow these instructions to add an SSH key. Next, set up a drush site alias. Finally, adjust the Behat job to run drush @my.alias sql-cli.

CircleCI

If you have Drush site aliases, then at the CircleCI dashboard go to the project's permissions and add an SSH key. Next, add drush @my.alias sql-cli to the Behat job at .circleci/config.yml.

Alternative

Alternatively, upload a sanitized database dump somewhere and set up the DB_DUMP_URL environment variable so the job can download it.

For example: Travis CI db env var CircleCI database via environment variable

A sample implementation is to use Dropbox API. See this repository for further details.

2. The production environment's files directory

If you have a site alias, then add drush rsync @my.alias @self to the Behat job. Alternatively, use Stage File Proxy module.

drupal9ci's People

Contributors

acha5066 avatar byrond avatar dakkusingh avatar deviantintegral avatar fjgarlin avatar juampynr avatar m4olivei avatar mrdavidburns avatar q0rban avatar sharique avatar stuffandthings avatar visabhishek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

drupal9ci's Issues

Error reading Debian Buster package list

I started getting an error when running apt-get update -y in GitLab CI, using the drupal8ci image. I'm wondering if anyone else is seeing this, since it seems unrelated to GitLab (someone else was able reproduce the error when pulling the image from Docker directly):

Get:1 http://security-cdn.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Ign:3 http://cdn-fastly.deb.debian.org/debian stretch InRelease
Get:2 http://security-cdn.debian.org/debian-security buster/updates InRelease [39.1 kB]
Get:4 http://cdn-fastly.deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:5 http://security-cdn.debian.org/debian-security stretch/updates/main amd64 Packages [492 kB]
Get:6 http://cdn-fastly.deb.debian.org/debian buster InRelease [163 kB]
Get:7 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease [46.8 kB]
Get:9 http://security-cdn.debian.org/debian-security buster/updates/main amd64 Packages [1132 B]
Get:8 http://cdn-fastly.deb.debian.org/debian stretch Release [118 kB]
Get:10 http://cdn-fastly.deb.debian.org/debian stretch Release.gpg [2434 B]
Get:11 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages.diff/Index [9592 B]
Get:12 http://cdn-fastly.deb.debian.org/debian buster/main amd64 Packages.diff/Index [27.9 kB]
Get:12 http://cdn-fastly.deb.debian.org/debian buster/main amd64 Packages.diff/Index [27.9 kB]
Get:13 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages [31.7 kB]
Get:14 http://cdn-fastly.deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:15 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 Packages [7082 kB]
Reading package lists...
E: Could not open file /var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-amd64_Packages.diff_Index - open (2: No such file or directory)
ERROR: Job failed: exit code 1

Any idea what could be causing this to start happening all of a sudden? My GitLab CI had been working fine for months. I've tried clearing the runner caches, but no luck.

DocumentRoot directory doesn't exist

I'm getting this error inside of CircleCI:

 [1G  8/26 [========>-------------------]  30%[Exec] Running service apache2 start
Starting Apache httpd web server: apache2AH00112: Warning: DocumentRoot [/var/www/html] does not exist

Do I need to configure Apache to look at /opt/drupal/docroot?

Errors saving and restoring cache

I'm getting a few errors related to cache keys:

error computing cache key: template: cacheKey:1:11: executing "cacheKey" at <checksum "package.json">: error calling checksum: open /home/circleci/project/package.json: no such file or directory

Error computing cache key: template: cacheKey:1:11: executing "cacheKey" at <checksum "package.json">: error calling checksum: open /home/circleci/project/package.json: no such file or directory

Here's what the cache section of the config.yml file looks like:

# Download and cache dependencies
- restore_cache:
    keys:
      # "composer.lock" can be used if it is committed to the repo
      - v1-dependencies-{{ checksum "composer.json" }}
      # fallback to using the latest cache if no exact match is found
      - v1-dependencies-{{ checksum "package.json" }}

- run: composer install -n --prefer-dist

- save_cache:
    key: v1-dependencies-{{ checksum "composer.json" }}
    paths:
      - ./vendor
- restore_cache:
    keys:
      - node-v1-{{ checksum "package.json" }}
- run: yarn install
- save_cache:
    key: node-v1-{{ checksum "package.json" }}
    paths:
      - node_modules

I've made a few changes from the original:

# Download and cache dependencies
- restore_cache:
    keys:
      # "composer.lock" can be used if it is committed to the repo
      - v1-dependencies-{{ checksum "composer.json" }}
      # fallback to using the latest cache if no exact match is found
      - v1-dependencies-

- run: composer install -n --prefer-dist

- save_cache:
    key: v1-dependencies-{{ checksum "composer.json" }}
    paths:
      - ./vendor
- restore_cache:
    keys:
      - node-v1-{{ checksum "package.json" }}
      - node-v1-
- run: yarn install
- save_cache:
    key: node-v1-{{ checksum "package.json" }}
    paths:
      - node_modules

I don't really understand what the purpose of saving and restoring a cached version of these two files in this CircleCI testing environment is, considering the environments only live for a few minutes. Can I just get rid of these?

Dupe Image in README

In the circleci section, I think you meant to post a screenshot of github showing circleci tests being run. Instead, you have the "add projects" image twice.

Functional tests not working

By functional tests I mean those extending BrowserTestBase, that sit in the Functional directory. I get the error "cURL error 7: Failed to connect to localhost port 8080: Connection refused". In my phpunit.xml I have set the SIMPLETEST_BASE_URL to http://localhost:8080. It also doesn't work with the default part 80.

I'm no docker export but I think this means port mapping is not set up, or I'm using the wrong port.

This isn't really a problem since I use Behat for those kind of tests and it's working fine but I have a bunch of drupal console generated functional tests that were failing.

circleci & travis alignment & continuity

Just seen @juampynr recent push with Travis. Great work first of all.
https://github.com/Lullabot/drupal8ci/blob/master/dist/travisci/.travis/docker-compose.yml

In some ways I quite like that it is using Wodby images. I also use Wodby for production stack, and their dinky image size is always a plus (Alpine).

It does however mean that:

  1. should we re-jig drupal8ci to use wodby instead of drupal-apache (official docker) images? Wodby uses Alpine and therefore images are much smaller.
  2. should we re-jig circleci flow to closely align with travis?

Needs some food for thought!! What do you guys think?

Is this still supported?

I've run into several problems getting this to work with a Drupal 8 project. Both messages I've seen appear to be related to the configuration of the image. I got this error during "Checkout Code": mkdir: cannot create directory '/var/www/html': File exists and then I read one of the other issues suggesting this needed to be changed to /opt/drupal but when I changed all instances of var/www/html I got this error in "Setup and Run Unit and Kernel Tests":

    There are no commands defined in the "install" namespace.  
                                                             

โžœ                                                                                                                  
โžœ   Requested RoboFile `/opt/drupal/RoboFile.php` is invalid, please provide valid absolute path to load Robofile. 
โžœ                                                                                                                  
โžœ                                                                                  
โžœ   Robo is not initialized here. Please run `robo init` to create a new RoboFile. 
โžœ                                                                                  

Add support for GitLab CI

Here's a start. Currently the kernel tests and behat tests fail.

stages:
  - build
  - test

# Template for composer managed dirs.
.composer_paths: &composer_paths
  - vendor
  - web/core
  - web/modules
  - web/themes
  - web/profiles
  - web/libraries
 
## Build the drupal8ci testing environment.
drupal8ci:build:
  stage: build
  image: juampynr/drupal8ci:latest
  services:
    - mariadb:10.3
    - selenium/standalone-chrome-debug:3.7.1-beryllium
  variables:
    MYSQL_ALLOW_EMPTY_PASSWORD: 1
  script:
    - composer install
    - robo install:dependencies
  # It'd be nice to use cache, but it's proven unreliable.
  artifacts:
    paths: *composer_paths

## Job to run Unit and Kernel tests.
drupal8ci:unit_kernel_tests:
  stage: test
  dependencies:
    - drupal8ci:build
  script:
    - apache2-foreground&
    - robo setup:drupal || true
    - cp .circleci/config/phpunit.xml web/core/
    - mkdir -p artifacts/phpunit
    - chmod -R 777 artifacts
    - cd web
    - sudo -E -u www-data ../vendor/bin/phpunit -c core --debug --verbose --log-junit ../artifacts/phpunit/phpunit.xml modules/custom
  artifacts:
    paths:
      - artifacts/phpunit

## Job to run the update path and Behat tests.
drupal8ci:behat_tests:
  stage: test
  dependencies:
    - drupal8ci:build
  script:
    - robo setup:drupal || true
    - cd web
    # For a quick start, set the following environment variable to a URL that contains
    # a database dump. Alternativelly, give CircleCI access to your development environment
    # and use Drush site aliases to run `drush sql-sync`.
    - wget -O dump.sql $DB_DUMP_URL
    - ../vendor/bin/drush sql-cli < dump.sql
    - ../vendor/bin/drush updatedb -y -v
    - ../vendor/bin/drush config-import -y -v
    - apache2-foreground&
    - cp .circleci/config/behat.yml tests/
    - chown -R www-data:www-data /var/www/html/web/sites/default/files
    - vendor/bin/behat --verbose -c tests/behat.yml
  artifacts:
    paths:
      - artifacts/behat

## Job to check coding standards.
drupal8ci:code_sniffer:
  stage: test
  dependencies:
    - drupal8ci:build
  script:
    - vendor/bin/phpcs --config-set installed_paths vendor/drupal/coder/coder_sniffer
    - mkdir -p artifacts/phpcs
    - vendor/bin/phpcs --standard=Drupal --report=junit --report-junit=artifacts/phpcs/phpcs.xml web/modules/custom
    - vendor/bin/phpcs --standard=DrupalPractice --report=junit --report-junit=artifacts/phpcs/phpcs.xml web/modules/custom
  artifacts:
    paths:
      - artifacts/phpcs/phpcs.xml

## Job to check test coverage.
drupal8ci:code_coverage:
  stage: test
  dependencies:
    - drupal8ci:build
  variables:
    SIMPLETEST_BASE_URL: "http://localhost"
    SIMPLETEST_DB: "sqlite://localhost//tmp/drupal.sqlite"
    BROWSERTEST_OUTPUT_DIRECTORY: "/var/www/html/sites/simpletest"
  script:
    - robo setup:drupal || true
    - cp .circleci/config/phpunit.xml web/core/
    - mkdir -p artifacts/coverage-xml
    - mkdir -p artifacts/coverage-html
    - chmod -R 777 artifacts
    - cd web
    - timeout 60m sudo -E -u www-data ../vendor/bin/phpunit --verbose --debug -c core --coverage-xml ../artifacts/coverage-xml --coverage-html ../artifacts/coverage-html --testsuite nonfunctional modules/custom
  artifacts:
    paths:
      - artifacts

Clarify instructions in scripts/databases

I'm trying to understand the instructions in scripts/database/README.md.

What is the purpose of these steps? Should I create my own image at quay.io?

The readme says "Create a database dump of the database and save it to scripts/database/dumps/drupal8.sql.
This is clear.

"This implementation uses a public Docker registry at Quay.io."
Which implementation?

"Here is how to build and push a new image for the first time:"
Do I need to execute these steps? Shouldnt I give the image a unique name?

Could not scan for classes inside vendor/drupal/core/lib/Drupal/

Hi,

I have created a Drupal 8 BLT project following the documentation at https://blt.readthedocs.io/en/latest/creating-new-project/ and am now trying to run curl -L https://github.com/lullabot/drupal8ci/raw/master/setup-circleci.sh | bash to set up CircleCI. However when I do I get the following errors:

Could not scan for classes inside "/Library/WebServer/Documents/my-project/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Library/WebServer/Documents/my-project/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/Library/WebServer/Documents/my-project/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be afile nor a folder

When I look at my project's directory structure there is no vendor/drupal/core but there is/Library/WebServer/Documents/my-project/**docroot**/core/lib/Drupal/Component etc

Is there something wrong with my setup or does something need modifying on this CircleCi script?

Drupal 9 compatibility

Trying this out of the box into a Drupal 9 project brings a lot of issues.

It'd be great to update for Drupal 9 compatibility. Perhaps 9.x branch or similar. If we get this to work then maybe the repo could be renamed to drupal-ci or similar.

I've started some work on it on the Travis side, so I'll open a PR to address this. I'm not super familiar with CI tools so any feedback/comments are appreciated.

Replace Behat tests by Cypress.io ones

Cypress.io's developer experince is much better than Behat's. Among other things:

  • Does not neet Chromedriver.
  • Takes screenshots and records clips by default.
  • Provides an app to view tests in action and debug them LIVE.
  • Has great documentation.
  • Has a browser plugin to record steps and export them.
  • It's written in JavaScript, which makes them more accessible to front end devs too.

Installing drupal8ci moves existing composer dependencies to dev dependencies

After creating a new drupal-project and installing drupal8ci, some of my dependencies have moved to dev dependencies.

diff --git a/composer.json b/composer.json
index 00f1e64..4ded610 100644
--- a/composer.json
+++ b/composer.json
@@ -17,18 +17,24 @@
     ],
     "require": {
         "composer/installers": "^1.2",
-        "cweagans/composer-patches": "^1.6",
         "drupal-composer/drupal-scaffold": "^2.2",
         "drupal/console": "^1.0.2",
         "drupal/core": "~8.4",
-        "drush/drush": "~8.0|^9.0.0-beta8",
         "vlucas/phpdotenv": "^2.4",
         "webflo/drupal-finder": "^1.0.0",
         "webmozart/path-util": "^2.3"
     },
     "require-dev": {
         "behat/mink": "~1.7",
+        "behat/mink-extension": "v2.2",
         "behat/mink-goutte-driver": "~1.2",
+        "behat/mink-selenium2-driver": "^1.3",
+        "bex/behat-screenshot": "^1.2",
+        "cweagans/composer-patches": "^1.6",
+        "drupal/coder": "^8.2",
+        "drupal/drupal-extension": "master-dev",
+        "drush/drush": "~8.1",
+        "guzzlehttp/guzzle": "^6.0@dev",
         "jcalderonzumba/gastonjs": "~1.0.2",
         "jcalderonzumba/mink-phantomjs-driver": "~0.3.1",
         "mikey179/vfsstream": "~1.2",

Update underlying docker image or use third party one

While working on #79 it was obvious that a lot has changed since last year (as it should in our world).

  • Latest Drupal is now PHP8+ compatible, which means that recent sqlite versions aren't shipped with PHP.
  • Composer 2 is now more stablished everywhere and should be the default
  • Node 10 is unsupported

So the suggestion is to create a new image with the latest and brightest (for now) and change all instances of juampynr/drupal8ci:latest for that new one. Or to leverage an existing image which has what we need in it.

We can probably get an image with most of our needs out there, and then maybe add the latest LTS node and a few other useful tools to it.

The work here is mostly on the Dockerfile(9)s and then the image will need to be uploaded to dockerhub. Once this is done we can change and test the rest of the code.

Related issues/PRs:

CICD dependencies in composer.json

Hi again guys,
quick question, would it be a better idea to have CICD dependencies added directly to Dockerfile instead of composer.json?

see: https://github.com/Lullabot/drupal8ci/blob/master/setup.sh#L28

	composer require --dev \
		behat/mink-extension:v2.2 \
		behat/mink-selenium2-driver:^1.3 \
		bex/behat-screenshot \
		drupal/coder:^8.2 \
		drupal/drupal-extension:master-dev

I am proposing something similar to what I am doing in my fork of drupal_tests:
https://github.com/dakkusingh/drupal_tests/blob/master/Dockerfile#L42

This way, any project using the docker image, regardless of the project setup, can benefit?

Happy to hear pros/cons of each approach!

Create Composer 2 version for both Dockerfile and Dockerfile9

Is your feature request related to a problem? Please describe.
The current images only support Composer 1, and even with hirak/prestissimo, the speed of dependency calculation and download are not comparable with Composer 2.

Describe the solution you'd like
I think this images should support Composer 2, but keeping Composer 1 to avoid breaking compatibility with existing services using the current images.

Describe alternatives you've considered
The clearer workaround, meanwhile, is to update composer and remove hirak/prestissimo on the very first step with the following:

      - run:
          name: Update Composer to version 2
          command: |
            set -eux
            composer global remove hirak/prestissimo
            composer self-update

This should update the image to Composer 2, removing hirak/prestissimo, until there is an alternative image using Composer 2.

Additional context
After a few months, Composer 2 still throw some issues in projects with Composer 1 dependencies. One example are phpro/grumphp:0.16.
The upgrade from Composer 1 to Composer 2 is not trivial and easy in some cases.
There should be, in my opinion, a clear distinction between the images running composer 1, the current ones, and the new images running composer 2, to avoid breaking compatibility with existing projects.

AlreadyInstalledException

Hi,

I've been following the guide with CircleCI but am having issues with the robo jobs e.g. robo job:generate-coverage-report - in fact all the jobs create the same error:

In install.core.inc line 536:
                                                                               
  [Drupal\Core\Installer\Exception\AlreadyInstalledException]                  
  <ul>                                                                         
  <li>To start over, you must empty your existing database and copy <em>defau  
  lt.settings.php</em> over <em>settings.php</em>.</li>                        
  <li>To upgrade an existing installation, proceed to the <a href="default/up  
  date.php">update script</a>.</li>                                            
  <li>View your <a href="http://:default">existing site</a>.</li>              
  </ul>

Looking here acquia/blt#720 there seems to have been a similar issue which is due to the db connection using localhost rather than 127.0.0.1. Is there any advice on how to set this up (in settings.local.php?) if this is the correct resolution?

Evaluate if we can use the official Drupal image

@deviantintegral suggested using the official Drupal image.

I looked at the official image's Dockerfile and verified that it downloads Drupal core, which is correct for https://github.com/deviantintegral/drupal_tests but not for this repository because there is already a Drupal 8 project. This is the reason why I went ahead using a custom image. However, I see value in extracting as much as we can from the official image or even better: propose a change so it can be left out.

xdebug-2.5.5: Need a PHP version >= 5.5.0 and < 7.2.0 (found 7.2.3)

RUN pecl install xdebug-2.5.5     && docker-php-ext-enable xdebug
 ---> Running in 019ae6fea9f1
downloading xdebug-2.5.5.tgz ...
Starting to download xdebug-2.5.5.tgz (279,491 bytes)
.....done: 279,491 bytes
76 source files, building
running: phpize
Configuring for:
PHP Api Version:         20170718
Zend Module Api No:      20170718
Zend Extension Api No:   320170718
building in /tmp/pear/temp/pear-build-defaultuseruYfHQ7/xdebug-2.5.5
running: /tmp/pear/temp/xdebug/configure --with-php-config=/usr/local/bin/php-config
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for icc... no
checking for suncc... no
checking whether cc understands -c and -o together... yes
checking for system library directory... lib
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for PHP prefix... /usr/local
checking for PHP includes... -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib
checking for PHP extension directory... /usr/local/lib/php/extensions/no-debug-non-zts-20170718
checking for PHP installed headers prefix... /usr/local/include/php
checking if debug is enabled... no
checking if zts is enabled... no
checking for re2c... re2c
checking for re2c version... 0.16 (ok)
checking for gawk... no
checking for nawk... nawk
checking if nawk is broken... no
checking whether to enable Xdebug support... yes, shared
checking Check for supported PHP versions... configure: error: not supported. Need a PHP version >= 5.5.0 and < 7.2.0 (found 7.2.3)
ERROR: `/tmp/pear/temp/xdebug/configure --with-php-config=/usr/local/bin/php-config' failed
The command '/bin/sh -c pecl install xdebug-2.5.5     && docker-php-ext-enable xdebug' returned a non-zero code: 1
make: *** [build] Error 1

xdebug-2.6.0 works.

bhat-test failed, why?

Hello, thanks a lot for this quite nice tool.
I tried installing travis-ti and run it but got an error with bhat-test. Please check here:

https://travis-ci.org/miura/bise/jobs/359122775

here is the last part of the log. Were should I supply URL for wget? I guess I forgot something to set up?

0.01s$ cp .travis/RoboFile.php .
0.17s$ vendor/bin/robo $JOB
 [Filesystem\FilesystemStack] mkdir ["mariadb-init"]
 [Exec] Running wget in mariadb-init
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
 [Exec]  Exit code 1  Time 0.01s
 [notice] Stopping on fail. Exiting....
 [error]  Exit Code: 1 
 [error]    in task Robo\Task\Base\Exec 

Thanks a lot for your tool and will be looking forward to your advice!

Kota

Changes to upstream image causes CircleCI to break

It looks like the upstream Drupal image changed such that it's causing the CircleCI job to fail on master:

https://circleci.com/gh/Lullabot/drupal8ci/106?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

In the same way our site project that uses the juampynr/drupal8ci:latest image is failing.

I believe it has to do with this change, which changed the working directory among other things:

docker-library/drupal#176

Next CI tool?

I am inclined to try GitLab CI but any suggestions are welcome.

Run Travis CI in a Drupal Docker project

I have a Drupal 8 project running in a Docker. But I am having a doubt about how to make the CI works.
Normally, I know that the .travis.yml file should resides in the root of my repo, however my Drupal project resides in a webroot folder (See attached image).

Due to the fact that the .travis.yml file is not in the root of the repo, the build always failed.

Any help would be highly appreciated.
Screenshot 2021-01-16 at 2 59 55 AM

Multisite recommendations

I'm currently working on a way to alter the default installation to work with our multisite. Ill attempt to contribute back here, but if anyone has some recommendations before I start hacking away that would be very useful

Drupal Functional tests not running

Drupal Unit tests run fine however I'm having trouble running tests in the Functional namespace that extend BrowserTestBase. I am using the config provided in the demo D8 repo.

cURL error 7: Failed to connect to localhost port 80: Connection refused
This error is thrown. I'm pretty sure this is the test failing to connect to the db.

In phpunit.xml I've got the simpletest values set as
<env name="SIMPLETEST_BASE_URL" value="http://localhost"/>
<env name="SIMPLETEST_DB" value="mysql://[email protected]/drupal8"/>

This issue is the same as described here #35

Not too sure if the base_url is wrong as the DB connection values look correct.
This is running in Circle CI. Any help is appreciated.
Screenshot 2020-02-03 at 10 05 05

"Failed to connect to localhost port 80: Connection refused"

Hi all,

Having a problem with the gitlab pipeline. I'm getting the following error when running functional tests:

1) Drupal\Tests\wpp_events\Functional\EventsPageTest::testEventsPageViews
GuzzleHttp\Exception\ConnectException: cURL error 7: Failed to connect to localhost port 80: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

Would welcome any suggestions on where to start fixing this!

My gitlab-ci.yml file looks like the following:

stages:
  - build
  - test

# The cache strategy for the build job.
.cache_strategy_push:
  cache:
    paths:
      - vendor
      - web/core
      - web/libraries
      - web/modules/contrib
      - web/profiles/contrib
      - web/themes/contrib
      - drush/Commands

# The cache strategy for the test jobs.
.cache_strategy_pull:
  extends: .cache_strategy_push
  cache:
    policy: pull

# The template for Drupal9CI tests.
.drupal9ci_test_template:
  extends: .cache_strategy_pull
  image: juampynr/drupal8ci:latest
  stage: test
  services:
    - name: mariadb:latest
      alias: db
  variables:
    MYSQL_ROOT_PASSWORD: root
    MYSQL_DATABASE: drupal
    DB_DUMP_URL: https://raw.githubusercontent.com/welly/uoe-test-db/main/db.sql
  script:
    - mysql --version
    - sleep 10
    - echo "SHOW tables;"| mysql -u root -p"$MYSQL_ROOT_PASSWORD" -h db "${MYSQL_DATABASE}"
    - mysql -u root -p"$MYSQL_ROOT_PASSWORD" -h db -e "create database if not exists ${MYSQL_DATABASE}"
  dependencies:
    - drupal9ci:build

## Job to build the environment.
drupal9ci:build:
  extends: .cache_strategy_push
  image: juampynr/drupal8ci:latest
  stage: build
  script: robo job:build
  artifacts:
    untracked: true
    paths:
      - ./*

## Job to check coding standards.
drupal9ci:code_sniffer:
  extends: .cache_strategy_pull
  image: juampynr/drupal8ci:latest
  script: vendor/bin/robo job:coding-standards
  artifacts:
    paths:
      - artifacts/phpcs/phpcs.xml

## Job to run Unit and Kernel tests.
drupal9ci:unit_kernel_tests:
  extends: .drupal9ci_test_template
  script: vendor/bin/robo job:unit-tests
  artifacts:
    paths:
      - junit.xml
    reports:
      junit: junit.xml

## Job to check test coverage.
drupal9ci:code_coverage:
  extends: .drupal9ci_test_template
  script: vendor/bin/robo job:coverage-report
  artifacts:
    paths:
      - artifacts
  coverage: '/^\s*Lines:\s*\d+.\d+\%/'

Thanks very much in advance

Config validation in CI

I used Drupal9CI project 1-2 yrs and after using now, I feel that it quite matured now. It is much simpler to configure CI and tweak it as well. I've been using Acquia BLT so far, compared to BLT this tool is much simpler.
For me one thing that is still missing is the configuration validation. Once that is done, I can completely replace BLT with Drupal9CI.

One more suggestion, can we rename project DrupalCI as it works for Drupal 8,9, and 10 and will also work few more releases.

Annotations for code-standards

Hi,

Unless I'm making a mistake, there are no annotations for code-standards. Which means that when it fails, there's no report of the actual code-standards issues.

Would be nice to have :)

Cannot touch 'storage/testing.sqlite': No such file or directory

I ran the install script, and have made a few changes. This is what I got from the install:

# PHP CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-php/ for more details
#
version: 2
jobs:
  build:
    docker:
      # Specify the version you desire here
      - image: circleci/php:7.1-browsers

    steps:
      - checkout

      - run: sudo apt update # PHP CircleCI 2.0 Configuration File# PHP CircleCI 2.0 Configuration File sudo apt install zlib1g-dev libsqlite3-dev
      - run: sudo apt-get update
      - run: sudo apt-get install -y libfreetype6-dev libjpeg62-turbo-dev libpng-dev
      - run: sudo docker-php-ext-configure gd --with-freetype --with-jpeg
      - run: sudo docker-php-ext-install -j$(nproc) gd

      # Download and cache dependencies
      - restore_cache:
          keys:
            # "composer.lock" can be used if it is committed to the repo
            - v1-dependencies-{{ checksum "composer.json" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-

      - run: composer install -n --prefer-dist

      - save_cache:
          key: v1-dependencies-{{ checksum "composer.json" }}
          paths:
            - ./vendor
      - restore_cache:
          keys:
            - node-v1-{{ checksum "package.json" }}
            - node-v1-
      - run: yarn install
      - save_cache:
          key: node-v1-{{ checksum "package.json" }}
          paths:
            - node_modules

      # prepare the database
      - run: touch storage/testing.sqlite
      - run: php artisan migrate --env=testing --database=sqlite_testing --force

      # run tests with phpunit or codecept
      #- run: ./vendor/bin/phpunit
      - run: ./vendor/bin/codecept build
      - run: ./vendor/bin/codecept run

And this is what I'm currently running:

# PHP CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-php/ for more details
#
version: 2
jobs:
  build:
    docker:
      # Specify the version you desire here
      - image: circleci/php:7.4-node-browsers

    steps:
      - checkout

      - run: sudo apt update # PHP CircleCI 2.0 Configuration File# PHP CircleCI 2.0 Configuration File sudo apt install zlib1g-dev libsqlite3-dev
      - run: sudo apt-get update
      - run: sudo apt-get install -y libfreetype6-dev libjpeg62-turbo-dev libpng-dev
      - run: sudo docker-php-ext-configure gd --with-freetype --with-jpeg
      - run: sudo docker-php-ext-install -j$(nproc) gd

      # Download and cache dependencies
      - restore_cache:
          keys:
            # "composer.lock" can be used if it is committed to the repo
            - v1-dependencies-{{ checksum "composer.json" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-{{ checksum "package.json" }}

      - run: composer install -n --prefer-dist

      - save_cache:
          key: v1-dependencies-{{ checksum "composer.json" }}
          paths:
            - ./vendor
      - restore_cache:
          keys:
            - node-v1-{{ checksum "package.json" }}
      - run: yarn install
      - save_cache:
          key: node-v1-{{ checksum "package.json" }}
          paths:
            - node_modules

      # prepare the database
      - run: touch storage/testing.sqlite
      - run: php artisan migrate --env=testing --database=sqlite_testing --force

      # run tests with phpunit or codecept
      #- run: ./vendor/bin/phpunit
      - run: ./vendor/bin/codecept build
      - run: ./vendor/bin/codecept run

I had to add the install command to get php's gd library installed. I also changed the image to 7.4-node-browsers, because I needed a newer php version, and the yarn install command was failing without node.

This is the error message related to sqlite:

#!/bin/bash -eo pipefail
touch storage/testing.sqlite
touch: cannot touch 'storage/testing.sqlite': No such file or directory

Exited with code exit status 1
CircleCI received exit code 1

I'm not actually sure I need sqlite - I'm planning to run visual regression tests against my staging environments.

Avoid recommending curl bash piping

Piping curl to bash is an anti-pattern. Is there a way we could avoid it? Could we have this be a package that you composer require instead?

If there's not a way to avoid it, we should at least wrap everything this script does in a function and then execute the function at the end. That way if the stream is interrupted, the partial script isn't executed.

Increase the ImageMagic max image dimensions

When a test failure occurs, Behat tries to take a screenshot of the failure. A screenshot is captured for every step of the test, and then concatenated together into one big image. These large images can easily exceed the maximum image dimensions for ImageMagick. We should increase this to a very large value by default.

This value can be modified in /etc/ImageMagick-6/policy.xml, e.g.

  <policy domain="resource" name="width" value="10MP"/>
  <policy domain="resource" name="height" value="200MP"/>

Interactive command line interface

Based on #80 (comment), we could turn this project into an interactive command line interface instead of a set of shell scripts.

We could use https://github.com/spf13/cobra-cli as a command line framework. The main benefit that I see of it being written in Go is that Go gets compiled into a binary that we could distribute in releases. Developers would just need to download the binary and use it without having to install anything else to run it.

Reorganize the dist directory

dist is starting to grow and mix together different CI integrations. Right now looks like this:

image

There are files and directories there specific for each CI tool and other files and directories which are common to them. We should:

  1. Create a subdirectory per CI tool such as circleci, travisci, gitlab.
  2. Place each CI tools files in their respective subdirectory.
  3. Move common files (like the tests) to a common subdirectory.
  4. Adjust the installation scripts so a user can run a one line command to install a single CI tool.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.