uitml / springfield Goto Github PK
View Code? Open in Web Editor NEWProject "Springfield"
Home Page: https://uitml.github.io/springfield/
License: MIT License
Project "Springfield"
Home Page: https://uitml.github.io/springfield/
License: MIT License
I came across an edgecase when the SSH agent is not running when prepare-authentication.sh is executed. The key/pub files are created, but they will of course not be added to the agent. The files will only be added to the agent if they don't exist beforehand, so in this case the user will have to add them manually after starting the SSH agent and run prepare-authentication.sh again to configure k8s.
We should investigate deploying the standard k8s dashboard solution, which might make it easier for end users to deploy and monitor jobs, etc.
When a user executes prepare-authentication.sh
with a pre-existing key pair, we should detect if the permissions are invalid.
And if the permissions are invalid, we should inform the user in a "friendly" manner and help them fix the permission issue.
An alternative approach would be to fix the permission issue automatically, but I'm a bit reluctant about changing file permissions on existing files; they might be "invalid" for a reason.
For reference, the .ssh
folder needs to be 0700
and any file within that folder should be 0600
.
We need to provide good documentation on how to get started with using the cluster once it's setup. Ideally we should provide some useful examples that can be used a starting point.
Investigate if we can switch from manually managing NVIDIA GPU drivers by deploying the GPU Operator.
We need to give end-users the ability to authenticate in order to access the cluster to schedule jobs/services, and so on.
The best option seems to be relying on GitHub OAuth. Another alternative is to OAuth via source.uit.no, but there doesn't seem to be a way of adding an OAuth application to the organization there, only to a user.
Currently the Getting Started guide walks the end-user through using port-forwarding to access the storage proxy (namespaced SSH server), but this is no longer necessary due to exposing the service via NodePort. The documentation needs to be updated to reflect this. Essentially replace the port-forwarding part with how instructions on how to find the correct port number.
Add information to the Getting Started guide about how to run the configuration script to correctly configure k8s client tools. It should probably be added between the sections on installing the client tools and connecting to the cluster file storage.
We need to limit resource quotas on the cluster based on role (researcher, student, etc) in order to prevent someone from using e.g. all GPUs or all CPU cores.
To simplify end-user workflow, sshfs
can be used to map the cluster file storage as a "local" mointpoint. In other words, the cluster file storage will look like a normal directory to the end-user, which will greatly improve the overall workflow.
Consider automating this (to some extent) using a shell script.
To improve the bus factor, we should document everything needed to recreate the cluster setup, in excruciating detail. The work has begun, but there is a lot left to be done. This task should have top priority.
Everyone using the cluster will need access to a persistent cluster storage. Since several pods/jobs will access the same persistent volume claims, the solution needs support for ReadWriteMany
access mode.
Currently the most promising (and easy to deploy) approach seems to be use a solution based on NFS. There's a Kubernetes NFS external storage incubator project based on Ganesha that seems promising.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.