Comments (12)
Thanks! There is currently at issue that prevents us from building at O3. We're building with no optimization at the moment. That you for the analysis and numbers! We will work on this soon!
from bashport.
Hello Igor,
I don‘t think this problem is related to compiler optimizations, only.
During my tests I also compared IBMs tcsh to IBMs sh.
The tcsh as slow as the different bash version, nearly.
While running the test script with different shells, I put my eyes into SDSF.
One thing I saw is much higher SIO values when running the faster sh.
Bash and tcsh had much lower values
at SIO.
So the problem may IO related?
Thanks Mike
from bashport.
time to dig into that OPT bug i guess. Fun :)
from bashport.
@mgrossmann i wonder if you are always getting the builtins for echo and cut and not 'failing over' to the coreutils versions?
from bashport.
I can't tell you that.
How can I check?
But I think that it is actually not
about echo or cut. Tomorrow I will test another script again.
I noticed the performance problem with our build system. A number of bash scripts around make. The whole thing is very very sluggish.
Best, Mike
from bashport.
one thought - change the script to say /bin/echo (say) and see if the perf changes?
from bashport.
Will do, tomorrow.
Thank you.
from bashport.
not sure if you're interested @mgrossmann but I would expect rebuilding at -O3 will make a huge perf difference for bash. Most of our tools are built with -O3 but we hit problems with bash and temporarily turned off optimization. Not the simplest problem in the world to track down, but a binary search approach is what I've used in the past. Break the set of compiled files in half and build half at opt, half not. Keep repeating until you isolate the file that, building at OPT fails, then you've narrowed it down to that source file (or files?). Then you can try functions although that is trickier because the optimizer can inline functions causing problems.
from bashport.
Hello Mike,
I ran some more tests today.
Regarding the built-in functions topic, with enable -n echo
you can control whether the built-in function should be used or not. In fact, from my point of view, the effect is not significant.
Assuming that Rocket builds their V4 with optimisations enabled, I have carried out further tests.
I explicitly compared Rocket's V4 with "our" V5. I can see slight differences here.
But even these are not as significant as the differences between SH and BASH.
For me, it seems much more as if the "sub-shelling" is the root of the problem.
I have used a slightly adapted shell script here. Comparing out=`echo ...`
with out=`bash -c echo ...`
showed a surprising result for me. The call of bash -c
was faster than the one without.
Using SH in shebang and simple backticks for sub-shelling
#!/bin/sh
count=1
#enable -n echo
while [ ${count} -le 50 ]
do
out=`echo "Some test data" | cut -c1`
count=$((count+1))
done
Result:
real 0m1.692s
user 0m0.365s
sys 0m0.122s
Using SH in shebang and sh -c
in backticks for sub-shelling
#!/bin/sh
count=1
#enable -n echo
while [ ${count} -le 50 ]
do
out=`/bin/sh -c "echo "Some test data" | cut -c1"`
count=$((count+1))
done
Result:
real 0m1.224s
user 0m0.307s
sys 0m0.102s
Using SH in shebang and bash -c
in backticks for sub-shelling
#!/bin/sh
count=1
#enable -n echo
while [ ${count} -le 50 ]
do
out=`/bin/bash -c "echo "Some test data" | cut -c1"`
count=$((count+1))
done
Result (V4):
real 0m4.770s
user 0m0.745s
sys 0m0.248s
Result (V5):
real 0m7.992s
user 0m1.203s
sys 0m0.401s
Using BASH in shebang and simple backticks for sub-shelling
#!/bin/bash
count=1
#enable -n echo
while [ ${count} -le 50 ]
do
out=`echo "Some test data" | cut -c1`
count=$((count+1))
done
Result (V4):
real 0m8.112s
user 0m2.593s
sys 0m0.864s
Result (V5):
real 0m12.309s
user 0m4.288s
sys 0m1.429s
Using BASH in shebang and sh -c
in backticks for sub-shelling
#!/bin/bash
count=1
#enable -n echo
while [ ${count} -le 50 ]
do
out=`/bin/sh -c "echo "Some test data" | cut -c1"`
count=$((count+1))
done
Result (V4):
real 0m3.628s
user 0m1.018s
sys 0m0.339s
Result(V5):
real 0m5.038s
user 0m1.495s
sys 0m0.498s
Using BASH in shebang and bash -c
in backticks for sub-shelling
#!/bin/bash
count=1
#enable -n echo
while [ ${count} -le 50 ]
do
out=`/bin/bash -c "echo "Some test data" | cut -c1"`
count=$((count+1))
done
Result (V4):
real 0m7.461s
user 0m1.480s
sys 0m0.493s
Result (V5):
real 0m10.919s
user 0m2.475s
sys 0m0.825s
On the subject of optimizations or general contribution. Unfortunately, I can't set up a development environment on our system. If you could give me access to the development system mentioned yesterday, I could well imagine contributing.
Regards, Mike
from bashport.
Nice progress. Yes - we can set up an account for you. @IgorTodorovskiIBM can you cover that since I am out on business?
from bashport.
Sure!
@mgrossmann, can you send me a public ssh key and an 8 character userid at [email protected]?
You can use ssh-keygen -t rsa -b 4096 -C "email id" -m PEM
on Linux/Window/Mac to create it.
from bashport.
Sent you a mail, Igor.
from bashport.
Related Issues (20)
- #! used with script as command does not work.
- guidance on shells HOT 10
- download failure HOT 5
- Investigate HAVE_HASH_BANG_EXEC induced git bug
- bash raises protection exception in Jenkins groovy environment
- Process substitution with bash 5.2.0 (Build 551) HOT 7
- netstat not found from the bash shell HOT 2
- awk issue under zot bash HOT 1
- ctrl+arrow isn't handled HOT 1
- upgrade to at least 5.2.15 HOT 1
- bash "type ping" is not able to point to external link ping command. HOT 1
- how to work with existing rocket env vars for bash HOT 2
- Wait command different behavior in bash
- Process substitution issues when using /dev/fd HOT 3
- Remove setting _CEE_RUNOPTS, _TAG_REDIR_IN etc from .env
- Bash does not source users bashrc file when started by non-interactive ssh command
- Bash garbling output after stderr to stdout redirection and then piping to tee
- A question about bash pipe(|) HOT 2
- A question about bash pipe(|) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bashport.