Comments (12)
Good question. /bin/sh
has the design decision as we expect every Docker container to have it. This is albeit we're aware that Atlassian Bitbucket Pipelines supports both /bin/sh
and /bin/bash
(and IIRC prefers /bin/bash
if available).
A couple of questions for clarification:
- Can you share the path of
bash
in your scenario within the container? - Is only the
script
andafter-script
execution affected by it? (Copying files, caches and artifacts work?) - Can you provide an exemplary pipleines YAML w/ such a script to test with?
We can then figure something out how to better support the bash
use-case I'm pretty sure.
The second issue with the environment variables that are not present is less clear to me, perhaps if an exemplary script could show this as well would be helpful to write more.
from pipelines.
We're using ubuntu dockers as base images (but with bash as SHELL).
-
So path is
/bin/bash
-
Yes, no other issues
-
This should do it:
FROM ubuntu:latest
SHELL ["/bin/bash", "-c"]
docker build -t ubuntu-bash .
image: ubuntu-bash
pipelines:
default:
- step:
name: Build and test
script:
- source /etc/os-release
- echo "$PRETTY_NAME"
from pipelines.
Thanks. Is the PRETTY_NAME
parameter non-null in the good case and null/undefined in the reported issue?
from pipelines.
Yes, because the content of /etc/os-release
is:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
...
(I put the wrong file to source in the example, copy past error. Fixed that now, so please modify in your trials as well)
from pipelines.
Please find a temporary topic branch issue-17-bash
in action. Build phar files (choose any) will be kept for two days so they can be used for early user testing.
I target release version 0.0.66 for the new bash runner feature.
from pipelines.
Thank you for taking the time to fix this!
This did fix the example I posted.
Unfortunately it still doesn't work for our full system. So I've put together a new example :/
FROM ubuntu:latest
RUN echo "HELLO=YOU" > ~/.bashrc
SHELL ["/bin/bash", "-c"]
image: ubuntu-bash
pipelines:
default:
- step:
name: Build and test
script:
- echo "$HELLO"
- source /etc/os-release
- echo "$PRETTY_NAME"
Now the outcome is the same as from the docker command:
$ docker run ubuntu-bash echo "$HELLO"
<nothing>
$ ~/Downloads/pipelines.phar
+++ step #1
name...........: "Build and test"
effective-image: ubuntu-bash
container......: pipelines-1.Build-and-test.default.test
container-id...: 7e400d87b793
+++ copying files into container...
+ echo "$HELLO"
+ source /etc/os-release
+ echo "$PRETTY_NAME"
Ubuntu 22.04 LTS
However, inside the docker:
$ docker run -it ubuntu-bash
root@10dc7d3e3e46:/# echo $HELLO
YOU
So from within the docker all variables declared in the .bashrc file are present. Looks like the pipelines run inside the dockerfile. This might be a completely different concept than what this project is doing?
from pipelines.
docker run -it
on (effectively) ubuntu:latest
has CMD ["bash"]
and loads ~/.bashrc
before dropping into the shell prompt.
pipelines
runs ubuntu:latest
detached with an interactive shell (but /bin/sh
) to keep it running. Then it executes the step script via docker exec
in the now running pipeline container (this deviates from Atlassian Bitbucket Pipelines and allows keeping the container).
This might be a completely different concept than what this project is doing?
Not sure I understand. From your feedback I read that you'd like to have the environment configured by ~/.bashrc
to be loaded. I'll do some tests with that.
You can also inspect what pipelines does by executing with -v
(verbose), it shows the docker commands. Additionally with --keep
you can keep the container running and then figure out if with docker exec ... $ID/$NAME ...
you can already come close to what you're looking for. This would also help me to better understand I guess.
from pipelines.
@Timple Did some tests for ~/.bashrc
and I think it improves on your use-case:
It now seems valid to me that, when using /bin/bash
as script runner, to execute (source as in source - not dot - bashism) it if it exists at the very beginning of the step scripts.
Rationale is the history of the Atlassian Bitbucket Pipelines Plugin. Before February 2017, scripts were run in an interactive shell. The Atlassian Bitbucket Pipelines Plugin continued "to execute the .bashrc file as if run in an interactive non-login shell but it" then behaved "as a non-interactive shell" (ref).
shell: [interactive, non-login] | ~> # February 2017 [ref]
~> [non-interactive, non-login]
+ if [ -f ~/.bashrc ]; then . ~/.bashrc; fi # from: ref: GNU Bash: [Invoked as an interactive non-login shell]
+' if [ -f ~/.bashrc ]; then source ~/.bashrc; fi # source (bashism) asserts /bin/bash (over /bin/sh)
+" test "$0" = "/bin/bash" && if [ -f ~/.bashrc ]; then source ~/.bashrc; fi # test for bash runner (/bin/bash)
ref: GNU Bash: Invoked as an interactive non-login shell
Please continue to find a temporary topic branch issue-17-bash
in action. Build phar files (choose any) will be kept for two days so they can be used for early user testing.
I continue to target release version 0.0.66 for the new bash runner feature.
from pipelines.
@Timple if .bashrc is executed, would it then match all of your expectations? Because then I'd say from my end this ready and I'll prepare the next release.
from pipelines.
Awesome, this works! Thank you.
One item left (unrelated so I'll open up a new issue!
from pipelines.
Released in 0.0.66.
from pipelines.
Rationale is the history of the Atlassian Bitbucket Pipelines Plugin. Before February 2017, scripts were run in an interactive shell. The Atlassian Bitbucket Pipelines Plugin continued "to execute the .bashrc file as if run in an interactive non-login shell but it" then behaved "as a non-interactive shell" (ref).
I only now read this comment (was on phone yesterday). You did an awesome job investiging!
from pipelines.
Related Issues (20)
- 'image' required in service definition HOT 3
- Dockerfile for pipelines? HOT 5
- `step.clone-path` does not match current Bitbucket behavior HOT 4
- Support for `condition` HOT 4
- (Some) Anchors Do Not Work - Raises Invalid YAML HOT 12
- Support for 'docker' in 'options' HOT 9
- Failed to parse validated YAML file HOT 2
- Pipelines Fatal Environment Definition Error is puzzling HOT 11
- BITBUCKET_REPO_SLUG holds the `.git` extension HOT 6
- PHP Deprecated: trim(): ... no pipeline to run! HOT 7
- script non-zero exit status: 243 HOT 3
- pipelines: file parse error: 'image' required in service definition HOT 1
- coverage-checker.php deprecated notice HOT 1
- Parallel step support for fail-fast and steps properties HOT 3
- Validator fails on pipes in steps HOT 3
- Dealing with SSH stuff HOT 16
- Services support (e.g. mysql) HOT 5
- Default pipeline with parallel fails validation HOT 3
- Working with private repos HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pipelines.