Comments (9)
I had quite a few problems getting the force:lightning:test:run process to work in a docker container under gitlab ci. Just adding some notes to help others.
Under the hood the sfdx client uses the npm package selenium-standalone. (Version 5.7.1 for sfdx 6.0.16.) The --configfile file on the sfdx command line is passed to the selenium-standalone module as the options. So see this module for what you can define in the config file. additionally the following sections of the config file are used :
-
webdriverio is also passed to the npm webdriverio module as the options to the remote command.
-
outputDivId is used to wait for the test results. It defines what the test harness is expecting to see on the html page at the end of the testing.
You need to make sure java is avaliable in the path, as well as google-chrome in the docker container. The selenium-standalone npm package automatically downloads selenium and the chromedriver files, you don't need those.
I had a lot of problem trying to debug issues where things seemed to work but nothing happened.
The problem was that I couldn't see what the sfdx client was doing. To help i added a shell script in the path before the real java command. This ran java and added some logging to chromedriver as well as re-directing the output of the java command to a file (so i could see it). There doesn't seem to be any other way to grab this output of the command executed by the sfdx client, the stderr and stdout of the subprocess are not captured by the logger.
For example
/usr/local/bin/java:
echo RUNNING java -Djava.util.logging.config.file=config/logging.properties -Dwebdriver.chrome.verboseLogging=verbose -Dwebdriver.chrome.logfile=/tmp/cd.log $@
/usr/bin/java -Djava.util.logging.config.file=config/logging.properties -Dwebdriver.chrome.verboseLogging=verbose -Dwebdriver.chrome.logfile=/tmp/cd.log $@ > /tmp/java.log 2>&1
I got force:lightning:test:run working with headless chrome using the following config file:-
{
"drivers": {
"chrome": {
"version": 2.33,
"arch": "x64",
"baseURL": "https://chromedriver.storage.googleapis.com"
}
},
"requestOpts": {
"timeout": 100000
},
"webdriverio": {
"desiredCapabilities": {
"browserName": "chrome",
"chromeOptions": {
"args": [
"--headless",
"--disable-gpu",
"--no-sandbox"
]
}
}
}
}
Some notes:
-
The driver section was necessary as the default driver version 2.23 didn't support chrome headless.
-
Be careful with the chrome options. I added --snapshot but this caused the broswer load to fail as the option was not compatible with the google --remote-debugging-port flag (which is added automatically by selenium). Some other combination of options failed to start chrome.
-
In my case the tests that i implemented took a long time, if there is a timeout with testing then you don't get a timeout message. It just fails. So if things appear to be working but you don't get the expected output then it might be a timeout issue.
-
I had to add the --no-sandbox option above, this isn't a good idea, however i was having a problem with chrome running on a docker debian image. Chrome would fail to run and I would get the following error:-
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
There seems to be a relevant discussion here jessfraz/dockerfiles#65 . Although i can't quite follow the discussion it seems there are some permissions that are required on the docker and/or the kernel images.
from lightningtestingservice.
for anyone coming back to this comment by @ghost - make sure to use a string (not a decimal) for the version
value under the driver selected - i.e.
{
"drivers": {
"chrome": {
- "version": 2.33,
+ "version": "2.33",
"arch": "x64",
"baseURL": "https://chromedriver.storage.googleapis.com"
}
},
"requestOpts": {
"timeout": 100000
},
"webdriverio": {
"desiredCapabilities": {
"browserName": "chrome",
"chromeOptions": {
"args": [
"--headless",
"--disable-gpu",
"--no-sandbox"
]
}
}
}
}
from lightningtestingservice.
Regarding how to specify web-driver options, see here and here
@alderete-sfdc plans on moving some of these details from issues to an FAQ section.
May be first try ensuring that java is installed and on the path? Basically The command uses webdriverio which has jre as a pre-requisite.
from lightningtestingservice.
After using the setup you described @innovate42 (which is extremely helpful!) - I still seem to get the least helpful error message:
Invoking Lightning tests using ghenkins-ci...
ERROR: Cannot read property 'endsWith' of undefined.
I'm pretty sure I will have to figure out 2 things
- how to install
google-chrome
onto my docker image (seems like it isn't part of theapt-get
repositories) - how to make the logging more verbose, potentially using your suggestion for the the java override
It would be a lot more helpful if the CLI tool itself helped users debug a bit better
from lightningtestingservice.
Hi @maniax89 I've found that the endsWith error seems to mean that the command that sfdx-clie was trying to run has failed. The best bet would be to try and see the output - and the only way i've found to do that is to create a wrapper around java as mentioned above.
I've added the Dockerfile that we are using for the test builds here https://github.com/innovate42/dockerfiles/tree/master/sfdx-testenv
from lightningtestingservice.
endsWith error is a being thrown by sfdx cli while trying to log the actually error see here. Cli team knows about it and its being tracked under bug W-4442231.
When people have encountered it in context of lightning:test:run. It normally has been resolved by setting DISPLAY/Xvfb.
from lightningtestingservice.
I was able to successfully get things running with #46 (comment) after installing google-chrome-stable
It would be good to have this issue archived/turned into documentation so that others may benefit (as it was extremely helpful)
from lightningtestingservice.
@alderete-sfdc FYI for doc
from lightningtestingservice.
Finally got everything working; hope this helps others that come behind me. While trying to get aura component testing working with Gitlab CI/CD using a Ubuntu 14.04 Docker image, I had to do the following three key things to get it working:
- Install Java 8 - Instructions: https://askubuntu.com/questions/464755/how-to-install-openjdk-8-on-14-04-lts
- Install google-chrome-stable - Instructions:
npm install selenium-standalone@latest -g
and thenselenium-standalone install
- Add
selenium/standalone-chrome:latest
toservices
section ingitlab-ci.yml
config file. - Used the following LTS command and config file. I specifically had to to add remote host and port settings in both seleniumArgs and webdriverio; seleniumArgs is required for a check that verifies that Selenium is running and webdriverio ensures the remote Selenium service is used instead of the localhost instance as that one fails to run on Gitlab CI runner (at least for me).
Test Command: sfdx force:lightning:test:run -a auraTests.app --configfile config/lts-config.json
lts-config.json
{
"requestOpts": {
"timeout": 100000
},
"seleniumArgs": [
"-host", "selenium__standalone-chrome",
"-port", "4444"
],
"webdriverio": {
"desiredCapabilities": {
"browserName": "chrome",
"chromeOptions": {
"args": [
"--headless",
"--disable-gpu",
"--no-sandbox"
]
}
},
"host": "selenium__standalone-chrome",
"port": 4444
}
}
gitlab-ci.yml
image: billable/ci-sfdx:latest
services:
- selenium/standalone-chrome:latest
cache:
key: "${CI_COMMIT_REF_SLUG}"
paths:
- node_modules/
variables:
SELENIUM_REMOTE_URL: "http://selenium__standalone-chrome:4444/wd/hub/"
CI_TEMP_DIR: /tmp
SCRATCH_ORG_DEF: config/ci-scratch-def.json
FORCE_APP: force-app/main/default
stages:
- build
- test
- analysis
- packaging
# Environment Setup
before_script:
- sfdx --version
- sfdx plugins --core
- mkdir ${CI_TEMP_DIR}/sfdx-keys
- openssl enc -d -aes-256-cbc -in ./build/assets/server.key.enc -out ${CI_TEMP_DIR}/sfdx-keys/server.key -pass pass:${DEVHUB_SERVER_PASS}
- sfdx force:auth:jwt:grant --clientid ${DEVHUB_CONSUMER_KEY} --jwtkeyfile ${CI_TEMP_DIR}/sfdx-keys/server.key --username ${DEVHUB_USERNAME} --setdefaultdevhubusername -a DevHub
# Build Application
build:
stage: build
script:
- npm install
- npm run build
artifacts:
paths:
- ${FORCE_APP}/aura/baseStyle/baseStyle.css
- ${FORCE_APP}/labels/*.xml
only:
- develop
# Test Automation
test:
stage: test
script:
# Testing Environment Setup
- sfdx force:org:create -s -f ${SCRATCH_ORG_DEF} -a gitlab_build_${CI_JOB_ID}
- sfdx force:lightning:test:install -t jasmine
- sfdx force:source:push -f -u gitlab_build_${CI_JOB_ID}
- sfdx force:user:permset:assign -n Billable_Admin
- mkdir -p "./build/test-results"
# Code Quality
# UI Testing
- source ./build/enable-lts-debug.sh
- npm run test:aura:remote
#- npm run test:lwc
# Apex Unit Testing
- sfdx force:apex:test:run -d ./build/test-results -u gitlab_build_${CI_JOB_ID} -l RunLocalTests -r human -w 10 -c
- node scripts/coverage.js
#Testing Environment Teardown
- sfdx force:org:delete -u gitlab_build_${CI_JOB_ID} -p
coverage: '/^Total\sCoverage:\s+(\d+.\d+\%)$/'
artifacts:
reports:
junit: ./build/test-results/test-*.xml
only:
- develop
from lightningtestingservice.
Related Issues (20)
- test an init function
- how to tests when v.body is set HOT 7
- Question on `sfdx force:lightning:test:install -t jasmine` HOT 2
- prerequisites to write tests? HOT 1
- chain spyOn? HOT 2
- GAC when testing scenario 2 HOT 3
- Testing with namespaces and Labels HOT 1
- jasmineTests.app not working HOT 5
- Setting DateTime attribute using $T.createComponent gives internal server error HOT 3
- Reporting does not escape every attribute in Junit format.
- jasmineTests.app Failed - Definition does not exist HOT 4
- Test reporter not escaping results
- How to test component events when event and handler are in different components?
- Mocking the internal clock in jasmine
- LTS for blur component event HOT 1
- Request: Support JEST HOT 1
- Unable to install LTS with SFDX HOT 4
- lightning:test:run execution permission denied HOT 2
- Fails On Page Load w/ "Freeze Javascript Prototype" Enabled HOT 3
- Installing with sfdx causes a GACK HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightningtestingservice.