galaxyproject / admin-training Goto Github PK

View Code? Open in Web Editor NEW

60.0 31.0 62.0 126.35 MB

Galaxy Admin Training

Home Page: https://galaxyproject.github.io/

Makefile 3.94% HTML 0.50% Python 4.57% CSS 0.83% Shell 11.40% HCL 78.77%

usegalaxy admin training development toolshed cluster nginx supervisor ansible tools

admin-training's People

Contributors

Stargazers

Watchers

Forkers

gitter-badger blankenberg joe-of-all-trades stevekm rkmadsen clijeron gvlproject natefoo pbfrandsen jennaj scholtalbers jfallmann bgruening bigdataguru elixir-no-nels silenus092 ulises1229 mvdbeek tarydium nsoranzo gonzalovera petraea laurentfournier olgaerm nagoue eugene79 jvanbraekel tranchant lecorguille chrisbarnettster nuwang raonyguimaraes cganote eddug jdavcs hexylena artbio molbiounige vikash84 hnlmarcus poterlowicz-lab migueljulia shiltemann kpbioteam torfinnnome tomraulet telukir slugger70 eancelet cat-bro apetkau nvk747 gacgama daishaoxing noissee tithuytrang sanjaysrikakulam usegalaxy-eu mira-miracoli

admin-training's Issues

add sentry to monitoring slides

Add dropbox/owncloud to the storage training

Create a document defining the minimum set of tools and functionality for a usegalaxy.* server

Some defined here, we should refine.

https://galaxyproject.org/usegalaxy/#membership

make "what's newer than a year?" - latest changes in Galaxy and future path slidedeck

Compute Cluster Updates

any updates / fixes needed?
Record slides (automated or manual?)
Record training

Improve data libraries training

galaxyproject/training-material#1848

update the playbook for setting up the machines

the ISMB version of the playbook is slightly different and I think more up to date?

https://github.com/galaxyproject/dagobah-training/blob/2017-ismb/GATC-ansible/playbook.yml

ex-tool-management.md task 13

In Task 13 https://github.com/galaxyproject/dagobah-training/blob/2018-oslo/sessions/04-tool-shed/ex-tool-management.md

instead of source activate ~/.venv/bin/activate

we should have

source ~/.venv/bin/activate

Add ephemeris to install a couple tools

using ephemeris and yaml tool files and categories etc.
How to extract tools.yaml from running Galaxy

Deploy existing tools to CVMFS / use this in production in .*

Deploy existing usegalaxy.* common toolset to CVMFS and switch current (3) usegalaxy.* servers to load tools from that source

Add watch_tool_data_dir: true

Otherwise the tool data tables won't reload automatically during the reference data tutorial

update ftp exercise with minimal playbook example

maybe use a subset of https://github.com/galaxyproject/ansible-proftpd ?

add SFTP/FTPS?

Securely Installing Galaxy lecture

Several Galaxy members have been working with TrustedCI to create a template for a System Security Plan document that will provide guidelines for best-practice deployments that improve system security. Could we find an hour-long slot in the Jan 2021 admin training schedule to go over this document and help admins orient themselves with it?

Next Meeting

TZ dependent, or, maybe via chat. (We’re the irc loving admins after all). Tentative date for the next meeting: 3rd of March

todo:

send a doodle

Slack

I've created a slack (galaxyadmintraining.slack.com) and created channels for the current planned topics.

The plan is students can chat within those channels to easily keep the topics separate, and our helpers will monitor those channels.

Correct Video Captions

Since we're doing video for this training, they absolutely need to be captioned, and captioned correctly. We have participants from all around the world and some of us have strong accents or speak quickly, we want these videos to be intelligible to all.

Process

Once videos are uploaded to youtube, ~1-2h later we get the automatic captions. They're quite good but not perfect and need a lot of polishing. This is where we need YOUR help. I'll be transferring the text google produces into a google document. I need people to work on these, listening to the video, and ensuring that the content is correct. , , etc. We don't need to catch 100% of capitalisation issues, but we should get most.

We Need You

Steps:

Open the Video (link under each section.)
Make Corrections
- Listen to the contents, and wherever you have corrections, correct the text.
- You do NOT need to use suggestion mode.
- Check for and add missing sentence breaks (. ? !)
- Ensure "code" bits are correct (e.g. 'group_vars', not 'group vars')
- Fix capitalisation errors (i, galaxy, ansible)
- Fix spelling errors
- DO NOT Fix grammatical errors, Remove Uhh/Umm/other filler words.
When you’re done with a block of work
- Highlight it with a colour
- We don’t expect one person to do the whole thing unless they’re motivated, some of these videos are quite long.
When a video is completely corrected, comment on the GitHub Issue saying it’s done.
- One of us will check that the text is highlighted and remove the section from this document
- Helena or Saskia will put the text back to the video in YouTube.

Videos Needing Caption Corrections

Below is a list of videos that need their text checked, and the relevant portion of a google doc with the text. When you feel good about a section, please just colour code it so anyone else working on parts of the video won't duplicate work.

add more grafana monitoring slides

Consider `set paste` in vimrc

Since many copy/paste issues involved indenting issues.

Determine suitability of refgenie for reference data management

add ethercalc GIE exercise

Training Feedback

Just collecting this for reporting

Kudos to the GAT team for these tutorials. I wasn’t aware of the layers of complexity and knowledge embedded in Galaxy. What I can say is that I am learning vast amounts of intel through simultaneous video and tutorial repetition. The computing part is quite extensive and encouraging for numerous views repetitions in tandem with documentation. I really hope that this type of asynchronous training is preserved. (edited)
I really appreciate how complete and thorough each lesson is, the fact that you touch every major component of the platform is invaluable, the clarifications about security and possible “Disasters” help you foresee possible issues and be proactive. Also I love the schedule page (all the needed material is there easy to access and to reach) and you instructors, you make it all seem easy. (edited)
My favourite thing about the course so far is how much information/depth there is in the training material and how it is embedded in the overall Galaxy documentation. A lot of courses I've attended in the past on tech matters have had unconnected or trivial examples, as a result as soon as you hit the real world the wheels come off
I've been trying to go through these tutorials on my own but never taken the time (got stuck on Ansible and server set up). So having this course now is awesome as it pushes me to do everything in one go and I am always impressed by the GTN team for updating tutorials and remembering everything you've put together. (edited)
Congratulations for this course. It is just amazing. Three years ago I setup a Galaxy server but the information was scarce and the cluster/pulsar sections were difficult to set up properly, or just impossible. I notice a huge improvement since then.
Hello, I know its not quite over yet (but the finish line is in sight!), but I would just like to thank all of the training admins for their colossal effort in putting together all of the training materials (and their contributions to Galaxy in general), it is greatly appreciated! This has been a great course and I feel empowered and inspired to apply many of the things learnt here to our institutional Galaxy servers. It has also given me some great ideas for new ways to manage the server(s) and improve the user experience.
A particularly huge thank you to @hexylena for answering all my dumb questions - and so quickly as well!
Today i could not do much since we had a tendor bidding for buying the server for our institute. It was a very exciting day... Whenever we get the server ready, the first thing im gonna do is deploy the galaxy. 🙂 This workshop was beyond fantastic. Words are not enough to describe how thankful I am. I have learnt alot. This has been really huge for me. I wish that your dreams come true sooner than tomorrow. :))

How to do the equivalent of sudo systemctl edit --full munge in Ubuntu 14.04

Can somebody kindly let me know how to do the equivalent of sudo systemctl edit --full munge on Ubuntu 14.04.

Ubuntu 14.04 doesn't use systemctl.

post-training improvements

Based on the assumption that many of the admins will also be the to-go resource for their users we should provide materials for short session with advanced usage (scratchbook, visualize/display, history search/filtering + multiview, collection manipulations etc.)

For next iteration we should make sure everybody is using same (or very similar) deployment environment. Troubleshooting heterogenous laptops was unnecessary overhead. We should also make sure network is permissive (e.g. ssh port) and performing as well as the VMs for participants (e.g. FTP port).

write ephemeris slides

modify tool management exercise to use ephemeris

old: https://github.com/galaxyproject/dagobah-training/blob/2019-pennstate/sessions/04-tool-shed/ex-tool-management.md

Docker stuff

remove smalt exercise
update all slides
@erasche provide a playbook to @jmchilton

Participant selected topics

A list of more obscure topics participants are interested in

Already discussed

tool HTML whitelist
External authentication
Active Directory authentication -- also how this affects ownership and sharing datasets.
Reports webapp
Conda environments for installing Galaxy, for running jobs and handle tools dependencies
Backup best practices

Slides Needing Video Compatible Speaker Notes

Swap data manager exercise for ephemeris

We can do this, right @Slugger70? Currently it's all manual and would be awesome if it used ephemeris as well

Update storage exercise for ansible + in training materials

Just write down the steps needed with new playbook awesomeness and I can write the surrounding bits again.

update basic Galaxy server exercise, add minimal ansible playbook

https://github.com/galaxyproject/dagobah-training/blob/2019-pennstate/sessions/14-ansible/ex2-galaxy-ansible.md

Update pulsar tutorial to use an MQ

Bioblend Updates

any updates / fixes needed?
Record slides (automated)
Record training

Ansibilize Pulsar

+ add to training materials (can continue to add the blabla / formatting if you like since I'm fast at it)

Pulsar Updates

#96, galaxyproject/training-material#2199
Record slides (automated or manual?)
Record training

Ansible Intro Galaxy start command discrepancy

In the Ansible intro here:

https://github.com/martenson/dagobah-training/blob/master/advanced/001-ansible/ex1-intro-ansible.md

Instructions say to start Galaxy like this:

Once, that is complete you can start your Galaxy server to test it.
Start the server with `sh ./run.sh --pid-file=main.pid --log-file=main.log --daemon`

But in the roles/galaxy-tool-install/defaults/main.yml file, the following are set

#The Galaxy pid and log file names.
galaxy_pid_file: paster.pid
galaxy_log_file: paster.log

I think either the pid and log file names need to be changed here, or the command to start Galaxy should be:

sh ./run.sh --pid-file=paster.pid --log-file=paster.log --daemon

Ansible Galaxy Updates

any updates / fixes needed?
Record slides (automated or manual?)
Record training

GTN tests/automation

We’re doing this for EU/AU: https://github.com/usegalaxy-eu/workflow-testing/
Would love to expand for US

Helper Introduction Videos

Individual Introductions

We’d love it if helpers would record a (max) 30 second introduction (1920x1080 please, zoom is recommended)

Your name
Your institute / location
Your background

We need this by thursday evening if it is to be included, or a "sorry no" (that's fine.)

Gildas
Saskia
....

DM @hexylena (slack, gitter, matrix) whenever you've got it done

GAT Introductions

Several people will be introducing various parts of the GAT as well:

@martenson - Schedule
@Slugger70 - Slack - what it is, how to use it. How to use threads, how to use emoji reactions (sect 3.), why they're important.
@nsoranzo - VM spreadsheet introduction (Where is your VM, what are the pulsar VMs, how to connect with SSH with that info)
@cat-bro - Byobu (keybinds, osx issues?)
@hexylena - GTN & Admin Topic in the training materials (Slides / tutorials / other languages.)
@shiltemann - YouTube (materials in description, subtitles, chapters, speed controls?)

Update "updating galaxy" documentation for ansible

E.g. https://galaxyproject.github.io/training-material/topics/admin/tutorials/upgrading/slides.html#7 needs to be replaced/removed

housekeeping slide with removing .pyc can go, showing the upgrade process with ansible is really amazing because it's just "change version, update galaxy vars, run playbook".

We could consider picking out a commit on dev, and having them upgrade to that?

Remove migrated materials

upgrading, migrated to https://galaxyproject.github.io/training-material/topics/admin/tutorials/upgrading/slides.html

prepare cloudbursting slides

Unified look and feel

Colour schemes (American flag for US, Blue for EU, Green for AU)
Quota/Tool/Reference request forms - all linked
Common tools in same categories in same order

document client changes

galaxyproject/galaxy#5094

any updates / fixes needed?
Record slides (automated or manual?)
Record training

fix fork-related links in materials

there are some URLs left in the materials that link to either martenson/gvlproject forks/github pages - we should convert those to their proper branch/galaxyproject counterparts.