red-hat-storage/rlocker

Resource Locker project

This project is a Back-end service with API endpoints and web UI, for monitoring the resources created in different cloud providers, define their usage availability by locking/unlocking a resource.

Project Dependencies:

• rlockerservices - Services repository that the rlocker uses
• rlockertools - A Python based client interface to interact with the platform

Quick Start Video will be added soon

Project Architecture:

Django website with applications, account, api, dashboard, health, lockable_resource, rqueue

• account: Designed to add more functionalities to the built-in user app

  • management: Directory that includes other python files that could be used as python manage.py name_of_file
  • __ init __: Packaging this account directory (Django App)
  • apps: Application configurations
  • signals: A signal to create a Token object so each Account will have an associated Token
  • urls: Urls that are related to the Account, like user login
  • views: Views that are related to auth actions like login and logout (We do not enable registration from the WEB UI)

• admin_tools: Designed to add more administrative actions, such as importing and exporting the current status of lockable resources

  • __ init __: Packaging this account directory (Django App)
  • apps: Application configurations
  • urls: Urls that are related to the Admin tools
  • views: Views in order to display the different administrative tools

• api: Application Programming Interface in order to automate resource locking releasing via endpoints.

  • __ init __: Packaging this account directory (Django App)
  • apps: Application configurations
  • custom_permissions: For enabling only access for non anonymous users, a permission mechanism should be written in api application
  • serializers: Serializers Object for the api, to prepare JSON responses.
  • urls: Endpoints that are enabled after the api prefix
  • utils: Useful functions for the api application
  • views: Views where we will take actions in each endpoint, depending on the request method.

• dashboard: Designed to display a read-only dashboard for more centralized view of the lockable resources with more details

  • __ init __: Packaging this account directory (Django App)
  • apps: Application configurations
  • urls: Endpoints that are enabled for this application
  • views: Views where we will manipulate the data we'd like to display in the dashboard

• health: Designed to have a health check URL that returns a raw string, so it will be easier to check health of the entire platform

  • __ init __: Packaging this account directory (Django App)
  • apps: Application configurations
  • urls: URL endpoints of this specific application
  • views: Views of this specific application

• lockable_resource: Application Programming Interface in order to automate resource locking releasing via endpoints.

  • __ init __: Packaging this account directory (Django App)
  • management: Directory that includes other python files that could be used as python manage.py name_of_file
  • action_manager: Includes all the classes that are doing some action on the lockable resource
  • admin: Model Registrations to the admin panel of the Django Application
  • apps: Application configurations
  • constants: Constants of the lockable resource app, we define there rules for available status types that a lockable resource object can have
  • exceptions: We define in the exceptions the actions that are not making sense, for i.e locking a resource that is already locked
  • label_manager: Takes responsibility to perform smart retrieving of an available resource by only specifying it's label
  • query_param_manager: Includes functions for handling query params in the lockable resource page (for i.e: ?view_as=yaml)
  • models: Modelize a schema of the table for different Lockable resources we will manage.
  • signals: Signals to be triggered once a specific field of lockable resource object changes
  • urls: Available URL's from the Web UI to manage the lockable resources
  • views: Design of the views and actions to take in GET/POST requests.

• rqueue: Application for managing the requests that could not be locked immediately, this is a queue management system application

  • __ init __: Packaging this account directory (Django App)
  • admin: Model Registrations to the admin panel of the Django Application
  • apps: Application configurations
  • constants: Constants of the rqueue app, we define there priority enumerations, time intervals for resource availability check
  • context_processors: Since we want some data to be accessible from each HTML template, we have to define it as a global context that is available (from settings.py)
  • models: Modelize a schema of the table for different requests in queue
  • signals: Signals to be triggered once a rqueue object is being created.
  • urls: Available URL's from the Web UI to manage the requests in queue
  • utils: Helper functions for the rqueue applications. for i.e - displaying nicer string for the pended time of a queue
  • views: Design of the views and actions to take in GET/POST requests.

How to use Addons (Plugins) in the Resource Locker Project (To be documented)

Dev mode:

• The following are commands to run in your terminal, each line represents a pattern of: command to run Explanation
  • git clone https://github.com/red-hat-storage/rlocker.git Clones the project
  • cd rlocker Change directory
  • python --version Python3.8 or above is required, please double-check this with that command
  • python -m venv venv Create a virtual environment for the django app
  • The following command differs from Windows to Linux systems:
    • venv\Scripts\activate Run this to activate the venv on Windows
    • source venv/bin/activate Run this to activate the venv on Linux
  • pip install -r requirements.txt Install the required packages
    • Expected errors on the command above:
      • psycopg2-binary==2.8.6 could not be installed
        • Optional solutions: (This package could be skipped in dev mode, do not hesitate to comment it out with #. The packages will be divided into dev packages and prod packages in the future)
          • Helpful SO poll, check out the pre-reqs according to your OS
  • The following commands are environment variable setups. The command to set an env variable is export (PLEASE USE SET AND NOT EXPORT IN WINDOWS!)
    • Visit the following website in order to generate a secret key for our Django application: link
      • Click the Generate button to generate a DJANGO_SECRET. Now the secret key is copied to your clipboard (Could be pasted the next time you send a Ctrl+V)
      • export DJANGO_SECRET='<YOUR_COPIED_SECRET>' Set this env variable. The wrapping with single quotes is important!
      • export USE_DEV_DB=True Here we specify to use the local db.sqlite3 file and not a PostgreSQL database engine (That is why, we could skip the psycopg2-binary package in previous steps)
      • export DEBUG=True Debug mode on.
  • Launch the website on the desired port (default is 8000)
    • python manage.py runserver <OPTIONAL_PORT> As said above, you do NOT HAVE to mention the <OPTIONAL_PORT>
  • Login Credentials are: admin, Admin-1
• Next Steps:
  • Visit the rlockerservices project in order to setup the queue_service. The queue service is the component that processes the queues that are coming from rlocker-cli.

Production

• To be documented

rlocker's Issues

API Docs

It would be nice to have some basic API docs exposed through the rlocker web interface.

Django-rest client and openapi have tools for this, which should make it easy to do.

Problems when doing 100 lock/release operations

While evaluating ResourceLocker for the LNST team. I wrote up a test to do 100 lock operations on the same resource to test the system behavior.

My test code had a wrapper are rlock command that did the lock and release operations in N threads.

We noticed that at some point in the process, one of the signoffs would get "stuck" and have to be released manually, and the process would continue fine.

Tested with 10,20,50, 75 of the same operations and it seemed to work fine.

Its entirely possible the issue is with my multi-threaded test code.

We tried to debug this a bit, but it was a little difficult given each lock/release operation was happening in its own python thread and it was hard to follow output.

We decided that perhaps it would be easier to test debug using a native python API so we are going to work on that and come back to this at a later time.

Resources in Maintenance mode are showed as Free

After recent change, when Enter Maintenance mode functionality was moved to sub-menu Actions, it is quite hidden the fact, that some resource is in Maintenance mode.
See on the following picture, the three resources in the pink rectangle. The only clue is the wrench and screwdriver icon on the Actions button, which is not too much visible.

Would it be possible to reflect the Maintenance mode in the Status column? For example like this:

• FREE
• LOCKED
• MAINTENANCE
• LOCKED - MAINTENANCE

When the resource is both locked and in maintenance mode, I think it make sense to reflect both states, but when it is not locked but in maintenance, I think the correct description is only maintenance.

When queue in retry mode, the time_finished box show the "completed" first time

After adding the mechanism of threading in queues, some queues might try to lock the same resource in the same timestamp.
So some queues are considered as in retry mode

But it still displays in a pending queue page the time that took from initializing until it turned status ALMOST_FINISHED. Therefore we need to continue counting. (Change the logic how the field time_pended_descriptive receive it's value)

Browser search functionality doesn't work correctly on Resource Locker web UI

If I want to search for some particular resource on the Lockable Resources page (Ctrl+F in browser) - for example DC-ECO - it shows, that there are X occurrences of that string on the page and based on the text beside the search field, it jumps over them when I click to next button (hit F3 key), but it didn't actually scroll the page to the particular place. I have to scroll manually and look for the highlighted text...
I tried it both in Firefox and Chrome with the same result.

Resources not freed when making a large number of request

While evaluating rlocker for use by the LNST team, we discovered that sometimes there appear to be issues with a large number of requests.

I developed a script that allowed me to do a large number of simualtanious lock/release operations and found when I did 10,20,50, 70 unlock/release operations on a single resource, the resource was always released and locked as expected. However, when I tried to do 100 operations we found that sometimes one of the signoffs would not get releases.

I did some debugging in order to get to the bottom of this, however I wasn't able to get to anything conclusive.

I some scripts that I will share that does 2 things:

1. provide a context manager based wrapper around rlockertools around locking/releasing a resource, albit it is still a work in progress when it comes to error handing.
2. Does the test and collect data able the operations.

You should be able to use these as a base to debug this. At some pojnt I think the context manager based wrapper would be a useful addition to rlockertools once it is more fully fleshed out.

Requests in queue does not show the number of days

In each queue pending / finished references, the queues are not showing the number of days that are waited.

We do not want to display the number of days for queues that are pending less than 24 hours.

But when we reach 24 hours, for example, 26 hours of waiting now, we cannot display 2 hours.

Besides, it should be 1 day, 2 hours.

No automation for changing the status of the queues

It will be useful to create a management command to manage the status of a group of queues.
The most common scenario is to abort all the pending queues. So we can create a new file inside rqueue/management/abort_pending_queues.py directory, that will just abort the pending queues.

And this way, it could be called within the directory of the project in this way:

python manage.py abort_pending_queues

FileNotFoundError during build

I am encountering the following FileNotFoundError when attempting to build the project using python 3.9.

$ python -m build ./
Found existing installation: setuptools 56.0.0
Uninstalling setuptools-56.0.0:
  Successfully uninstalled setuptools-56.0.0
Collecting setuptools>=40.8.0
  Downloading setuptools-57.2.0-py3-none-any.whl (818 kB)
     |████████████████████████████████| 818 kB 1.9 MB/s
Collecting wheel
  Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools
Successfully installed setuptools-57.2.0 wheel-0.36.2
WARNING: You are using pip version 21.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/private/var/folders/cm/qrs4d_9j4dq5j6hj3snm72qm0000gn/T/build-env-jmk0eefv/bin/python -m pip install --upgrade pip' command.
running egg_info
creating rlockertools.egg-info
writing rlockertools.egg-info/PKG-INFO
writing dependency_links to rlockertools.egg-info/dependency_links.txt
writing entry points to rlockertools.egg-info/entry_points.txt
writing requirements to rlockertools.egg-info/requires.txt
writing top-level names to rlockertools.egg-info/top_level.txt
writing manifest file 'rlockertools.egg-info/SOURCES.txt'
reading manifest file 'rlockertools.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'rlockertools.egg-info/SOURCES.txt'
running sdist
running egg_info
writing rlockertools.egg-info/PKG-INFO
writing dependency_links to rlockertools.egg-info/dependency_links.txt
writing entry points to rlockertools.egg-info/entry_points.txt
writing requirements to rlockertools.egg-info/requires.txt
writing top-level names to rlockertools.egg-info/top_level.txt
reading manifest file 'rlockertools.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'rlockertools.egg-info/SOURCES.txt'
running check
creating rlockertools-0.3.9
creating rlockertools-0.3.9/framework
creating rlockertools-0.3.9/rlockertools
creating rlockertools-0.3.9/rlockertools.egg-info
copying files to rlockertools-0.3.9...
copying LICENSE.txt -> rlockertools-0.3.9
copying README.md -> rlockertools-0.3.9
copying setup.py -> rlockertools-0.3.9
copying framework/__init__.py -> rlockertools-0.3.9/framework
copying framework/main.py -> rlockertools-0.3.9/framework
copying rlockertools/__init__.py -> rlockertools-0.3.9/rlockertools
copying rlockertools/exceptions.py -> rlockertools-0.3.9/rlockertools
copying rlockertools/resourcelocker.py -> rlockertools-0.3.9/rlockertools
copying rlockertools/utils.py -> rlockertools-0.3.9/rlockertools
copying rlockertools.egg-info/PKG-INFO -> rlockertools-0.3.9/rlockertools.egg-info
copying rlockertools.egg-info/SOURCES.txt -> rlockertools-0.3.9/rlockertools.egg-info
copying rlockertools.egg-info/dependency_links.txt -> rlockertools-0.3.9/rlockertools.egg-info
copying rlockertools.egg-info/entry_points.txt -> rlockertools-0.3.9/rlockertools.egg-info
copying rlockertools.egg-info/requires.txt -> rlockertools-0.3.9/rlockertools.egg-info
copying rlockertools.egg-info/top_level.txt -> rlockertools-0.3.9/rlockertools.egg-info
Writing rlockertools-0.3.9/setup.cfg
Creating tar archive
removing 'rlockertools-0.3.9' (and everything under it)
Found existing installation: setuptools 56.0.0
Uninstalling setuptools-56.0.0:
  Successfully uninstalled setuptools-56.0.0
Collecting setuptools>=40.8.0
  Using cached setuptools-57.2.0-py3-none-any.whl (818 kB)
Collecting wheel
  Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools
Successfully installed setuptools-57.2.0 wheel-0.36.2
WARNING: You are using pip version 21.1.1; however, version 21.1.3 is available.
You should consider upgrading via the '/private/var/folders/cm/qrs4d_9j4dq5j6hj3snm72qm0000gn/T/build-env-nfm5wd4k/bin/python -m pip install --upgrade pip' command.
Traceback (most recent call last):
  File "/Users/clacroix/workspace/rlockertools/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 280, in <module>
    main()
  File "/Users/clacroix/workspace/rlockertools/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 263, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/Users/clacroix/workspace/rlockertools/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 114, in get_requires_for_build_wheel
    return hook(config_settings)
  File "/private/var/folders/cm/qrs4d_9j4dq5j6hj3snm72qm0000gn/T/build-env-nfm5wd4k/lib/python3.9/site-packages/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
    return self._get_build_requires(
  File "/private/var/folders/cm/qrs4d_9j4dq5j6hj3snm72qm0000gn/T/build-env-nfm5wd4k/lib/python3.9/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
    self.run_setup()
  File "/private/var/folders/cm/qrs4d_9j4dq5j6hj3snm72qm0000gn/T/build-env-nfm5wd4k/lib/python3.9/site-packages/setuptools/build_meta.py", line 258, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/private/var/folders/cm/qrs4d_9j4dq5j6hj3snm72qm0000gn/T/build-env-nfm5wd4k/lib/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 6, in <module>
    with open("HISTORY.md") as history_file:
FileNotFoundError: [Errno 2] No such file or directory: 'HISTORY.md'

ERROR Backend subproccess exited when trying to invoke get_requires_for_build_wheel

Plugin Dev info

As far as I know the rlocker expiry plugin is the only plugin out there right now.

It would be nice to have more docs and info and examples about what is required to create and deploy a plugin. In the future my team might want to create some to offer team specific features or UI.

Sort resources alphabetically by name

The resources on Lockable Resources page seems to be sorted by ID, would it be possible to sort them by name? It will be easier for searching and mainly similar resources will be together in a group.

[RFE] Resources by label statistics page (all/free)

Would it be possible to create new page, where will be list of labels and information about number of all resources with that label and number of actually available resources?
There is a similar section Free Resource By Label on the dashboard page, but it doesn't have the information about all resources with that label.
Also I think, that compact (alphabetically sorted by label) table will be the best format for this.