Comments (4)
Please could you describe in detail exactly what behaviour you would expect? Specifically:
- What you believe should happen if an instance is started with
--no-redirect
, and the terminal where it was started is then closed. - If the instance should still run in the background. This would mean that any output would appear in the terminal in a way which interferes with any other work that continues there.
Due to the way instances are implemented, it is not necessarily trivial to implement a --no-redirect
.
Because you can simply tail -f
the log files that the instance writes to, it's unlikely that this feature will be added unless it is requested by a lot of users, or the code changes are contributed in a PR by a member of the community.
from singularity.
Thank you for your reply. As for your concerns:
- The main idea of my intention is that the behavior meets common users' expectation. That is, the output should be what the commands display when they are run normally on Linux instead of in a container. So, specifically, I hope Singularity can forward the output to both the terminal and log file like
tee
, no matter the terminal is still there or closed. Since the gist is for the behavior to meet a normal user's expectation, if the terminal is closed by the user, the user should not expect to see any output in a terminal that does not exist.
I agree that --no-redirect
is a confusing name of the option, which suggests not redirecting output to log file. What I want is instead forward the output to both terminal and redirected file, so maybe --tee
is a better name, because it is reasonable to assume that most Linux users know what tee
command does.
- For this concern, we can simply respect the behavior of the commands in the %startscript section when they are run normally in Linux (not in container), because that meets users' expectation. If the commands output something on terminal that interferes with the current work that continues there, the user should expect that effect and does not complain, because that is the native behavior of the command. The duty of Singularity is not regulate or improve the existing (bad) behavior of commands, but truthfully mimic the behavior of the command. Just let it be if interference happens, as we often see when a booting just finishes (some remaining messages of services interfering with login prompt).
But a good news is that the interference is not that common. What users want to see (or expect to see) is usually a banner of the software (like name and version), confirmation message that the service is started successfully or failed, as well as some important configuration information like port being listened. Such a message are stdout and is output to terminal only at the beginning of the service. Likewise, if the service exits, there could be some stdout message showing that the service exits successfully, as well as possibly what resources are released. Both these cases will not interfere with terminal. What you are concerning about is the output of service in the middle of other work running on terminal. But if there are any such messages, it is highly likely these messages are errors which should be displayed in stderr, not in terminal (aka stdout) to cause any interference. Of course, there could still be stdout messages in the middle from a service, like a web server which reports requests it receives. But again, that's the behavior of the command when it's running normally in Linux (not in container). The user should expect that interference and hence no complaints. For cases like this, I usually leave the terminal open and open a new terminal to do other jobs. Or, if the user really wanna disable output, s/he can remove the --tee
option I suggested above. You can set the default to be no --tee
.
I think forwarding messages destined to stdout to both stdout and a file is not hard. It is an average exercise of Unix programming, cf. e.g., "Advanced Programming in the Unix Environment", Third Edition. By W. Richard Stevens and Stephen A. Rago, Addison Wesley. 2013.
from singularity.
Thanks for the detail.
- The main idea of my intention is that the behavior meets common users' expectation. That is, the output should be what the commands display when they are run normally on Linux instead of in a container.
Our expectation of how Singularity is used is that singularity run
and the %runscript
would be used to get this behaviour. Starting an instance is, instead, explicitly for something in the background, long running and detached, that you don't interact with directly.
Note that you can do a singularity run mycontainer.sif &
as a kind of half-way approach.
I can see your point for some cases, but I don't think it's necessarily the most common expectation for an instance. For those who might read this and are also familiar with Docker, instances are like starting a docker container without -it
... there is no direct output. You have to inspect the logs.
We can leave this issue open to see if it attracts support from others, or if anyone is interested in contributing the code change to implement the feature.
I agree that
--no-redirect
is a confusing name of the option, which suggests not redirecting output to log file. What I want is instead forward the output to both terminal and redirected file, so maybe--tee
is a better name, because it is reasonable to assume that most Linux users know whattee
command does.
Agreed ... --tee
might be a good one.
I think forwarding messages destined to stdout to both stdout and a file is not hard. It is an average exercise of Unix programming, cf. e.g., "Advanced Programming in the Unix Environment", Third Edition. By W. Richard Stevens and Stephen A. Rago, Addison Wesley. 2013.
Unfortunately it's not as easy dealing with stdout / stderr of containers as simple processes. The container has to start in the background, completely detached from the terminal, with some kind of attachment to the log files, that will be maintained at all times. Then, thesingularity
process in the foreground has to transition into a process that attaches to the container stdout / stderr to show it on the terminal also. When we setup and start a container, there are several processes involved, across multiple exec boundaries, namespaces etc. There is a lot of complexity. The create container->start container in background->attach from CLI
pattern is what's implemented by all the container runtimes that support background containers similar to Singularity instances, and offer the ability to display the output to a terminal.
We actually have some code that does pieces of this flow for the OCI-mode (but not the native runtime / instances) e.g. here: https://github.com/sylabs/singularity/blob/main/internal/pkg/runtime/launcher/oci/oci_attach_linux.go
For instances, we currently just replace the Stdout / Stderr file descriptors with file descriptors for log files. There is no persistent process redirecting output through a buffer, nor a socket that we can attach to - so it's actually a considerable amount of work needed to implement this. https://github.com/sylabs/singularity/blob/main/internal/pkg/instance/instance_linux.go#L271
from singularity.
I'd close it.
from singularity.
Related Issues (20)
- Support OCI-SIF data-container in --bind & --mount
- Support squashfs->tar in push of an OCI-SIF data container
- Remove CentOS 7 from CI matrix:
- Remove EL7, SLES 11 examples
- Remove Yum CentOS7 bootstrap tests
- Remove CentOS 7 regression test for issue 5250 HOT 1
- Remove CentOS 7 install specifics:
- Remove kernel version as proxy requirement for OCSP test
- Remove EL7 test case / handling of old ld listing HOT 1
- oci: fixed descriptor capacity for OCI-SIF prevents pulling some images
- singularity build --fakeroot fails with free(): invalid pointer while spawning RPC server HOT 3
- Pip install uses too much disk space - error with /tmp as tmpfs HOT 2
- Unknown image format/type in nextflow pipeline HOT 5
- Drop remaining direct containers/common usage
- CNI dhcp plugin does not work - netns bind issues
- oci-sif: inefficent copies of oci (layout) and tarball images
- delete sandbox
- allow user ns in singularity.conf
- allow ipc ns in singularity.conf
- e2e: OCI CustomHomePreservesRootShell failure
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from singularity.