Giter Site home page Giter Site logo

ocf / ocfweb Goto Github PK

View Code? Open in Web Editor NEW
38.0 47.0 88.0 6.15 MB

The main ocf website

Home Page: https://www.ocf.berkeley.edu

License: Other

Makefile 0.50% Python 60.06% HTML 29.52% JavaScript 2.32% CSS 1.30% Shell 0.16% Dockerfile 0.49% SCSS 5.48% Jinja 0.17%
django scss ocf bootstrap docker pre-commit jenkinsfile makefile dockerfile

ocfweb's Introduction

ocfweb

Build Status Code Health Checked with mypy

The main ocf website.

Working on ocfweb

We recommend following all of these steps on supernova, the staff login server, because it is already configured to run ocfweb in development mode with minimal extra setup.

Clone the repo, and be sure to check out submodules:

$ git submodule update --init
$ make install-hooks

If you get an error about not being able to import bootstrap, it's because you forgot to run the second command.

Running in development mode

On supernova[0]run make dev. The first time will take a while, but future runs will be almost instant thanks to pip-faster.

It will start listening on a deterministically random port (really, 8000 plus the last 3 digits of your user id) which is printed to you. You can then view the site in development.

Building SCSS

Run make scss to build SCSS. You can also use make watch-scss to rebuild it automatically when SCSS files change.

Running tests

To run tests locally, run make test. Please don't push to master with failing tests—Jenkins will refuse to deploy your code, and nobody will be able to deploy until it's fixed.

If you make a pull request to the OCF GitHub organization from your fork of ocfweb, Jenkins will attempt to build and test your branch automatically. If your build fails, you can log into Jenkins to see which tests you've failed and fix them, if running make test locally didn't already tell you.

You can run individual tests with venv/bin/pytest -k <test_name> or venv/bin/pytest <test_file>::<test_name> if running all tests is too slow.

Running pre-commit

We use pre-commit to lint our code before commiting. While some of the rules might seem a little arbitrary, it helps keep the style consistent, and ensure annoying things like trailing whitespace don't creep in.

You can simply run make install-hooks to install the necessary git hooks; once installed, pre-commit will run every time you commit.

Alternatively, if you'd rather not install any hooks, you can simply use make test as usual, which will also run the hooks.

Almost all build failures of ocfweb can be tied to something pre-commit probably would have caught.

Installing packages

To install a package to the production environment, add it to requirements-minimal.txt, then run make update-requirements. Similarly, to install to the development environment, add to requirements-dev-minimal.txt and run make update-requirements. Use as loose a version requirement as possible, e.g. try django or django>=1.10,<1.10.999 before django==1.10.0.


[0]: If you have a staff VM, you can also use that to run ocfweb. You will have to add the ocf_ocfweb::dev_config class to your Hiera node config. Specifically, add:

classes:
    - ocf_ocfweb::dev_config

It's probably easier to just run everything on supernova.

ocfweb's People

Contributors

abizer avatar asaiacai avatar baisang avatar ben9583 avatar bjb568 avatar bzh-bzh avatar cg505 avatar chriskuehl avatar daradib avatar dependabot[bot] avatar dkess avatar ethanhs avatar fawaf avatar fydai avatar ja5087 avatar jvperrin avatar kennydo avatar kpengboy avatar matthew-mcallister avatar mcint avatar mrminos avatar naderm avatar nickimpicciche avatar oliver-ni avatar pre-commit-ci[bot] avatar saurabhnarain avatar singingtelegram avatar tahabi avatar tliu22 avatar vio-eli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ocfweb's Issues

Mobile dropdown-menu slightly too small.

The max-height for the navbar is slightly smaller than the height of all the menus added together. This allows a small confusing margin of scrolling of several pixels. By increasing it or removing the max-height, this can be solved.

Port /OCF/ to ocfweb

Per first BoD discussion we don't want to launch this with the top bar links leading off-site.

Currently only "Join" (atool) and "About Us" (/OCF/) lead off-site. Porting atool before launch is not practical, but porting at least the base /OCF/ probably is.

(There is still #2 for porting the rest of the /OCF/* pages.)

Status icons should be more intelligent

Currently they are just constant True, which is only ok for the "web" one.

We can change them by pushing the site, but ideally they'd do something smart (even pinging the relevant server would be OK).

It would be cool if they read a file in ~staff so that we could update the status immediately.

Header anchor link causes h4 to jitter in docs

When you hover over an h4 header in docs, the anchor link icon which appears is taller than the text, so the height increases causing the page to jitter a tiny bit. We should decrease the icon size or increase the text size or something since it's a little annoying.

Display recent update(s) from status blog

This may or may not be tricky, but even getting only the headlines from the status blog would be really valuable. We want everyone to be well aware what's happening when there's a security alert or a service goes down.

Docs: explain off-site vs. OCF hosting

A common question, brought up in rt#3973. Should probably distinguish free vs. proprietary software, off-site vs. on-site, web software versus web hosting service (and how the latter two are often intentionally muddled up). Also give a somewhat more complete list of available software.

replace references to hearst gym with MLK

I can do this tonight if somebody doesn't grab it first.

The Google Map is duplicated and yucky right now because I copy-pasted it to support two locations. We should merge those back into a single location (we won't need to support two locations again in the near future).

Responsive layout on docs

It's ok now, it could use a few tweaks (e.g. ToC appears at the end, probably should just omit entirely)

Mention user quota somewhere

I just raised quotas to 5 GB, but can't find any mention anywhere of the old quota to update. We should probably mention it somewhere...

Merge atool into ocfweb

This is a long-term goal and will take some work. Actually merging the code should be straightforward. The main concerns are deployment:

  • Need to add login somehow to ocfweb (not hard, just needs some thought; we want to handle both logging in with CalNet, and logging in with an OCF account)
  • Need to ensure we treat cookies carefully
    • invalidating sessions on privilege change to avoid session fixation -- which atool doesn't do currently
    • ensure cookies don't get passed to user sites

It would be cool if it could use GSSAPI for users in the lab (and e.g. feature their print quota on the home page).

Wiki pages, staff hours, home page should use some caching

Currently loading the "documentation" page results in every single wiki page being loaded from disk and Markdown-d. It's easy to cache this.

Additionally, the home page makes a bunch of LDAP calls which we can cache.

The Django cache didn't look very useful for function-level caching (which is what we want), maybe we want to use functools.lru_cache or something. Need to be a little careful about dev though.

Site goes down when we deploy

There are a few seconds when the site is down when we deploy. Might be tricky to fix but would be a cool goal. Occasionally it triggers monitoring when the timing is unlucky.

Add easy way to post urgent messages

On the current website we sometimes post urgent messages like this:

It would be nice if we could do something similar on ocfweb, preferably reading from an external file so we don't have to push the site to update it.

It should also allow overriding the "is lab open" check (which currently is based on ocflib hours).

favicon is pretty blurry

I made it from the SVG directly (i.e. not resizing a raster), so not sure why. Probably needs some pixel love.

get_blog_posts sometimes returns None which isn't handled properly

didn't investigate at all yet

A problem was encountered and reported via ocflib:

An exception occured in ocfweb:

Traceback (most recent call last):
  File "/usr/share/python/ocfweb/lib/python3.4/site-packages/django/core/handlers/base.py", line 147, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/share/python/ocfweb/lib/python3.4/site-packages/ocfweb/main/home.py", line 24, in home
    blog_posts = get_blog_posts()[:2]
TypeError: 'NoneType' object is not subscriptable



Request:
  * Host: www.ocf.berkeley.edu
  * Path: /
  * Method: GET
  * Secure: True

Request Headers:
{'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
 'HTTP_CONNECTION': 'close',
 'HTTP_HOST': 'www.ocf.berkeley.edu',
 'HTTP_USER_AGENT': 'Rackspace Monitoring/1.1 '
                    '(https://monitoring.api.rackspacecloud.com)',
 'HTTP_X_FORWARDED_FOR': '78.136.44.23, 169.229.226.23',
 'HTTP_X_FORWARDED_HOST': 'www.ocf.berkeley.edu',
 'HTTP_X_FORWARDED_PROTO': 'https',
 'HTTP_X_FORWARDED_SERVER': 'www.ocf.berkeley.edu',
 'PATH_INFO': '/',
 'QUERY_STRING': '',
 'RAW_URI': '/',
 'REMOTE_ADDR': '127.0.0.1',
 'REMOTE_PORT': '46784',
 'REQUEST_METHOD': 'GET',
 'SCRIPT_NAME': '',
 'SERVER_NAME': '127.0.0.1',
 'SERVER_PORT': '8000',
 'SERVER_PROTOCOL': 'HTTP/1.0',
 'SERVER_SOFTWARE': 'gunicorn/19.4.1',
 'gunicorn.socket': <socket.socket fd=16, family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8000), raddr=('127.0.0.1', 46784)>,
 'wsgi.errors': <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7efe33be1908>,
 'wsgi.file_wrapper': <class 'gunicorn.http.wsgi.FileWrapper'>,
 'wsgi.input': <gunicorn.http.body.Body object at 0x7efe33be1208>,
 'wsgi.multiprocess': True,
 'wsgi.multithread': False,
 'wsgi.run_once': False,
 'wsgi.url_scheme': 'https',
 'wsgi.version': (1, 0)}

Session:
{}


====
Hostname: coma.ocf.berkeley.edu
Callstack:
    at /usr/share/python/ocfweb/lib/python3.4/site-packages/ocflib/misc/mail.py:59 (send_problem_report)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/ocfweb/middleware/errors.py:47 (process_exception)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/django/core/handlers/base.py:262 (process_exception_by_middleware)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/django/core/handlers/base.py:149 (get_response)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/django/core/handlers/wsgi.py:177 (__call__)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/workers/sync.py:171 (handle_request)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/workers/sync.py:130 (handle)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/workers/sync.py:30 (accept)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/workers/sync.py:66 (run_for_one)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/workers/sync.py:119 (run)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/workers/base.py:126 (init_process)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/arbiter.py:515 (spawn_worker)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/arbiter.py:548 (spawn_workers)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/arbiter.py:482 (manage_workers)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/arbiter.py:179 (run)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/app/base.py:72 (run)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/app/base.py:192 (run)
        by /usr/share/python/ocfweb/lib/python3.4/site-packages/gunicorn/app/wsgiapp.py:74 (run)
        by /usr/share/python/ocfweb/bin/gunicorn:11 (<module>)

Figure out how to deploy in prod

My thoughts for deploying right now:

  • Package as a Debian package (dh-virtualenv), similar to atool or create; install on death
  • Run under gunicorn, supervise with systemd (just like atool)
  • Proxy to gunicorn from Apache if the request doesn't match a userdir pattern.

Yet unanswered questions:

  • How do we serve static assets? Currently we deploy them to https://www.ocf.berkeley.edu/~ocfweb/static/ but it's not a great solution.

    Right now I'm thinking package them and deploy them to some directory Apache serves at static.ocf.berkeley.edu or similar.

  • How do we gracefully fall back to the old content? We can set up Apache to proxy if it doesn't match a userdir pattern, but this means old pages like /OCF/whatever will be going to ocfweb, and it needs some way to handle them if it doesn't know about the page.

    There aren't that many pages on our current site, maybe we can just hard-code them in the Apache config as we remove them one-by-one (so we never actually proxy to ocfweb for them). Here's a list of current pages: https://gist.github.com/chriskuehl/8be24760c01e76574dfd (some are not actual pages)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.