preloading jobe (or jobeinabox maybe) with some files

[trampgeek]

preloading jobe (or jobeinabox maybe) with some files

[trampgeek]

When you say "into the environment", do you mean "into the temporary working directory for the run"? If so, then the short answer is probably no, since the only files that get loaded into that directory need to be loaded from the filecache, by their unique ID. However, you could have Jobe run a Python script/program instead of a Java program; the script could set up the environment, copying files from anywhere you choose to put them in Jobe, then compile and run the Java code using the subprocess module. That gives you complete flexibility for other tasks like pre-run checks on the Java code, setting up a directory tree for packages, etc.

But do your jar files need to be in the current working directory? Could you not put them in a fixed part of the file system, say /usr/local/share, and refer to them with a full path name?

[gigamonkey]

When I asked the question I think I meant "into the temporary working directory" but my understanding was more fuzzy than it is now—I got jobe running in Docker and ran a few test jobs last night. And indeed they don't need to be in the current working directory—my whole idea was to have a set of classes that a lot of jobs would need access to so the nicest thing would be to have one copy on the jobe server that jobs could add to their classpath as needed.

I guess I had assumed that the individual jobs were running in a chroot jail or something and therefore wouldn't be able to access arbitrary files. But maybe that's not the case?

[trampgeek]

Jobe tasks run as normal unprivileged users on a Linux system except there's no home directory and there are tight limits on resource usage, such as cpuseconds, number of processes, disk output, memory use, .... The working directory in which they run is inaccessible to all other users - there's one user per task.

[gigamonkey]

Ah got it. Now that you say it I feel like I read that sentence somewhere in the docs but it didn't quite sink in. And I assume the users who run the jobs don't have privileges to write outside their working directory so they can't scribble over anything important?

[trampgeek]

Correct. They have minimal access to anything else except /tmp, which is purged after the run. Unless a sysadmin chooses to create world-writeable directories.

[gigamonkey]

Thanks for you help. Closing this to keep you project tidy. Cheers!