Giter Site home page Giter Site logo

usepa / haztrak Goto Github PK

View Code? Open in Web Editor NEW
42.0 42.0 18.0 19.77 MB

An open-source web app that illustrates how waste management software can interface with RCRAInfo to track hazardous waste electronically

Home Page: https://usepa.github.io/haztrak/

License: MIT License

Python 33.01% HTML 0.30% Shell 2.75% TypeScript 56.78% Dockerfile 0.44% JavaScript 3.95% SCSS 2.76%
django olem reactjs

haztrak's People

Contributors

ayobamiseun avatar dependabot[bot] avatar dpgraham4401 avatar nirajpdn avatar wiljnichepa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

haztrak's Issues

Migrate to bootstrap 5

Do this for fun, not because it absolutely needs it. Will take time, but I expect not a huge mental lift.

Migrate from the Bootstrap Framework version 4.6.1 to 5.x.x as the title says. This would be nice to get done before the front end becomes more than a placeholder. Right now (4/15/2022), I don't think migrating would be that difficult because we're not heavily using any bootstrap features or javascript.

There is a little JQuery being used but it's minimal, located in the static content directory.

haztrak's frontend starting point was SBadmin2
Bootstrap's documentation on migration to version 5

reinforce test suite

๐Ÿž Bug Report

Setup a more robust test suite for the haztrak client and server.

I'm leaving this open ended for anyone interested in contributing, this is a good way to get better acquainted with the code base, and we'll accept PR with even a small number of (quality) tests.

  • #191
  • Add error boundaries to HtCard and the features
  • Test HtCard catches all errors with generic error
  • Add Test for Yup schemas
    • Handler form Schema Test
    • Transporter Search Schema Test

Manifest Form Validation

Break down the (e-)Manifest form into components (already started), each component should have accompanying validation (see below)

These dependencies are already available in Haztrak

client side validation is handled with yup, react-hook-form has some basic validation built it, currently we're keeping that logic in yup validation schema. The entire manifest form can be broken down in to reusable components (e.g., address validation/form for all handlers should be the same).

  • #111
    • note: TSD and transporter info should generally match what's in RCRAInfo (for hazardous waste manifest), since this is an example application we should keep it so that information is pulled from RCRAInfo whenever available
  • #110
    • lots of work that can be done here
  • #109
    • basic stuff, like potential ship date should be greater than current date

Build out the manifest views/forms/actions

Summary

  • users should be able to use the below assets/actions

View Manifest

  • Just take the e-Manifest view, upgrade the view look (using cards)
  • View should indicate which site you are viewing the manifest as

Edit Manifest

  • Bonus point criteria
  • Fields that RCRAinfo does not allow to be updated should be locked
  • manifests locked by regulators will be uneditable

Draft Manifest

  • the ability to make new manifests from scratch
  • manifest is not assigned an MTN until POSTed to RCRAinfo in Pending status

React Site Dashboard

Goal

Replicate the current Site Dashboard as a react component

Frontend

  • build out bootstrap UI for sites that user has access to.
  • Routing (require authentication)
  • Slice

Backend

  • Convert Django (apps.trak.views) view to Django-rest APIView
  • request needs to be authenticated

Basic Documentation

If the goal of this project is to build something that others can (possibly) adopt or use as an example, this project's success will live or die by it's documentation

  • organize the repo to make it welcoming? putting all source code in separate directory???
  • Docs directory
    - I don't think we want to use GitHub Wiki in case other look to migrate to other git hosting services
  • models ERD (I believe can use something like pgadmin4 when using postgres)
  • #32
  • #33

Unit Testing

build out a basic suite of unit and integration test that should be run (at least!) before commits are merged into the main branch. May want to consider using git hooks for pre-commit test as well. See issue on building out a basic CI/CD pipeline

Basic CI/CD pipeline

It would be great to create a quick CI/CD pipeline that should do a couple things

  • Run a test suite before commits are merged into the main branch, prevent if tests do not pass
  • increment the version based on the commit messages
    • this would require using pre-commit hooks that enforce a commit message style (e.g., fix:, feat:, MAJOR:)
    • apply this version to the git tags
  • automatically produce a change log from previous version

create Address model

pretty self-explanatory title,
will require some work with the custom serializer create methods, this will likely not be a straightforward task but should be easier to implement and maintain than the janky way it's in there right now.

Convert Manifest view to APIView

As the title suggest

May start off with a basic APIView, would be good to migrate to one of Django's generic views maybe sometime down the line

Site search API

๐Ÿš€ Feature Request

This is likely more difficult than I imagine. we either need to...

  1. redirect site searches to the RCRAInfo API
    • implement a Restful API endpoint that accepts search criteria for a RCRAInfo site
    • the API would redirect that site search criteria to RCRAInfo
    • the final response from the new haztrak endpoint should redirect the RCRAInfo response
  2. find a way for haztrak users to use the RCRAInfo site search API directly from the client
    • Goes without saying, the front end should not have access to the user's RCRAInfo API ID and Key since there's not a good way to secure the user's API ID and Key in the front end.

Something that pings a third party API seems like something that should use a messaging broker like Redis. I haven't decided if that makes sense for the scope of this project. I'm not sure what that would take to implement it since I' haven't research this.

Refactor LayoutSidenav_nav

๐Ÿš€ Feature Request

Refactor the navigation in the side column to a react sub-component. For example right now the Site and the Help drop down navigation are almost identical except for the links and texts, changes in the appearance of one would not be shown in the other.


screenshot_22-11-04-16:03

Redo Site Manifest view to display 1 table with multiple views.

๐Ÿš€ Feature Request

We'd like to be able to display more rows of manifest than what can be displayed with 3 tables in this view here.

instead of displaying 3 separate tables (The manifest affiliated with this site as a hazardous waste generator, transporter, and TSD), show 1 table with 3 views (that can be navigated via links/buttons at the top). Each view should have one of the three tables.

Default the view to Generator for now.

Manifest Pagination

๐Ÿš€ Feature Request

When loading a site's manifests, manifests should be paginated. This is probably a good first issue for someone looking to get experience with React. No dependencies should need to be pulled in, but a custom hook will likely need to be built.

freecodecamp.org has a blog that may be useful as a guide.

With regards to defaults, I would say we should display ~10 manifest on site's page. We may need to reorg a Site Manifest view so it only displays 1 table (on which the view can be switched from generator, transporter, TSD) instead of all three. when applicable, being displayed in three rows. See #132

'Add Transporter' section to New Manifest

๐Ÿš€ Feature Request

This is a high level issue that covers the sub-issues that will be needed to implement a React component used for searching the haztrak http server for existing transporters and adding the transporter to a manifest being edited.

Depends on

Error handle bad login request

๐Ÿž Bug Report

end goal

When user is not found or error occurs at login, display the error (bonus points for looking good while doing it).

description

There is still a lot of prototype code in the login function and redux user slice in general. right now, when a user attempts to login and uses bad credentials or the haztrak server cannot be reached, an error is displayed by redux in the console but that's it.

This the login.rejected reducer function needs to be updated from

state.error = action.payload.error; // <-- The payload object is undefined when the promise is rejected

to

state.error = action.error.message; // <-- a string

Data Flow Documentation/Diagrams

๐Ÿš€ Feature Request

Is your feature request related to a problem? Please describe.

  • Documentation (preferably with visual aids) showing the business logic (steps) Haztrak uses when interfacing with RCRAInfo.
  • This is a very high level task, no code required, but would be good to do before we start diving into the meat of the project (interfacing with RCRAInfo)

Describe the solution you'd like

  • the docs directory should contain a file/section on the workflow for actions that interface with RCRAInfo's web services.
  • example off top of my head
    • Title: Editing a manifest through the post-submission correction processing
    1. User Select edit manifest
    2. haztrak checks that the current version is up-to-date with RCRAInfo
    3. if not, it ask the user if they'd like to update with the most recent info before editing
    4. if yes, haztrak pulls the most recent manifest from RCRAInfo and serves to user for editing

Describe alternatives you've considered

  • we just wing it and don't plan this stuff out beforehand

Add haztrak 'logo' to login screen

๐Ÿš€ Feature Request

haztrak does not really have a logo, but for now, it would be good to add a label to the login screen.

We can dimply use the same 'logo' that's in the top left corner of the home screen (after login) found here

Profile: Add site

๐Ÿš€ Feature Request

Is your feature request related to a problem? Please describe.

  • Users need a way to add a new Site(s) to their Profile.

Describe the solution you'd like

When users add a Site to their profile, if that site corresponds to a Handler that exists/is a registered site with

Describe alternatives you've considered

When the user does not have a API ID and Key (or an admin that allows them to use their API credentials), it should notify them that they will not be using up-to-date site details.

Initial Profile view

๐Ÿš€ Feature Request

Implement a simple profile view for the front end at https://domain.com/profile There is already an feature folder with a Bootstrap card at client/src/features/profile. Currently, it shows the users RCRAInfo API ID and Key (also need to remove that information from being accessible through the API, that's a separate issue). Remove the rcrainfo API ID and Key from the card, and add a simple dispay which shows the user their profile information, which is delivered from the server via this object...

{
  "user": "string",
  "epaSites": ["string"]
  "phoneNumber": "string" | null
}

We had a basic example before haztrak was converted to a SPA design pattern here in the pre-release in html.

Create manifest in RCRAInfo.

๐Ÿš€ Feature Request

This is dependent on many other things being implemented first.

  1. Draft manifest
  2. everything that draft manifest needs to be implemented (all the manifest typescript definitions and form & validations)

This is a high level issue and may have many sub issues.

Migrate to Bootstrap 5.x

Migrate from the Bootstrap Framework version 4.6.1 to 5.x.x as the title says. This would be nice to get done before the front end becomes more than a placeholder. Right now (4/15/2022), I don't think migrating would be that difficult because we're not heavily using any bootstrap features or javascript.

There is a little JQuery being used but it's minimal, located in the static content directory.

Convert last function views/handlers to class based

I think the only views that remain as functional based views are those from the account django.app. It would be nice to have consistent approach (i.e., convert account views to class based view). This would be a good first issue.

dockerize server

Title says it all, low priority

  • bonus points for multistage build that reduces size
  • see gunicorm (or however it's spelled)

Update high level docs

๐Ÿš€ Feature Request

Update documentation with a high level overview of the application services here.

The docs should cover these points

  1. Services
    • React front end, Django http server, message broker (redis), task queue, database
    • supported databases (postgres)
  2. #65

Profile State Management

Idea

Once logged in, implement a React-Redux slice that manages state for user profile and contain information such as

  • Sites with access
    • permissions associate with those sites
  • e-Manifest relevant user info (contact info, legal name)

This could allow haztrak to do things such as

  • auto fill user details when singing
  • outline which sites a user can view/edit/sign as when looking at a manifest

integrations with e-Manifest/RCRAInfo

Alright, here we go, getting into the meat of things.

Goal

Create a series of endpoints that allow for users to request server side actions related to integrating with RCRAInfo (e.g., calling the RCRAInfo/e-Manifest API)

High level example

  • Logged in Haztrak user clicks button
  • Browser side JavaScript calls Haztrak API endpoint
  • Haztrak API uses the user's RCRAInfo API ID/Key to make request to RCRAInfo
  • Haztrak processes RCRAInfo response on server side
  • Return response to Haztrak user/browser with success/fail/relevant data

Pull Updates from RCRAInfo

  • pull manifests from RCRAInfo
  • this might be a challenge, it will require pulling the manifest, find diffs and applying those difference (for now, just applying to the same instance, in the future we may include manifest versioning)

Push Updates to RCRAinfo

  • see section title, e-Manifest does not allow PATCH or PUT, so just involves posting the full manifests

e-Manifest link (all sorts of stuff COULD be done with the UI link service)

For a single manifest, UI link service for RCRAInfo

  • Link to View
  • Link to Sign
  • Link to Edit
    For a Site, RCRAInfo UI link for
  • e-Manifest Dashbaord
  • Bulk Quick Sign

Manifest Form: General info

This Issue includes 2 main parts...

  • #146
    • Creating the React component with the all the necessary fields.
    • ask for this information if necessary
  • Manifest General Info form validation scheme
    • a reusable validations scheme via yup

Add Redis and Celery

๐Ÿš€ Feature Request

This is a relatively large change, It means that the haztrak api service has to be accompanied with a redis and celery service deployment as well.

While the added complexity is unfortunate for the scope of this project, using a task queue (along with the message broker) is necessary for an application that so heavily relies on an external service (RCRAinfo) to perform it's core functions (using e-Manifest).

Implement the manifest model and serializers

Haztrak needs to be able to consume and store manifest data from rcrainfo. At a high level this includes

  • pulling from rcrainfo (or third party)
  • deserializing to native python objects
  • how this data is represented in persistent storage

In addition, haztrak should be able to serve manifests as well

  • serializing to json
  • exposing an (RESTful) API

pulling is easy, that's already implemented by the emanfiest client library which pulls the manifests down as json.

Model and Serializer implementation:

  • created date
  • update Date
  • mtn
  • submission type
  • signature status
  • origin type
  • shipped date
  • potential ship date
  • received date
  • certified date
  • certified by
  • generator
  • transporters
  • tsd
  • broker
  • waste
  • rejection boolean
  • rejection info
  • import boolean
  • import info
  • contains rejection or residue
  • printed doc
  • form doc
  • additional info
  • correction info
  • PPC status
  • locked
  • locked reason

A lot of the fields lower on this list are lower priority and we will continue to simply take the json from RCRAInfo and store it as text.

Transporter and waste will be tricky ones and are the last thing to really implement before we can move on to other things.

In the future, the ability to store PDFs will be required but for now we're not focused on that.

Convert SiteView to APIView

As the title suggest

Should be fine with just a basic APIView with GET as allowed method (these are part of the Site Model, not handler, Site is specific to sites that the user has access to)

Ability to manage user profile

Users need to be able to manage their basic profile info

  • change profile data
  • recover password
  • change RCRAInfo API ID and key
  • see which EPA ID numbers they're affiliated with (awesome to be able to view your RCRAInfo permissions)

Most of this is complete, django docs should be able to help implement "see which EPA ID i have access to" and django auth_views has generic views for resetting passwords but email users may be a heavier lift. have not looked into it.

Transporter search endpoint (Haztrak DB only)

๐Ÿš€ Feature Request

Add an endpoint that will allow users to search for transporters that already exist in the haztrak back end. The new API endpoint should accept POST request only, and expect the body to include the following.

{
  "epaId": string,      // corresponding to the handler/transporter's EPA ID number
  "name": string        // corresponding to the handler/transporter name
}

The endpoint should return an array of transporters who's EPA ID number AND name contain the strings queried for via the API. The endpoint does not need to know about all possible tranporters in RCRAInfo, just those stored in the haztrak database.

Create mySite model that extends handler model

Right now the Handler and EpaSite model capture the same information. they are also now located in the same django app since the trak and site apps were consolidated.

Effort to remove duplication, we will reconfigure to create the following...
The Handler model is used to capture the generator, transporters, and TSD listed on a manifest.
The Site model has a one-to-one relationship with the handler model so with django, having access to a Site will give access to all information capture in the handler model and will and can also be extended to include information not needed by RCRAInfo such as

  • waste streams
  • haztrak access and permissions
  • anything the haztrak Site needs that's not

Organize front end (manifest) routing

This is a 2 part questions. 1 how users can navigate to their manifests in haztrak? How do we organize the routing itself? Answering this may help wrap out noggins around the number of React features we're going to need.

  1. It would be nice to be able to access a users manifest without having to drill down so many clicks from home -> site -> site manifests -> manifest view/edit.

    • Home may include dashboard info on manifests at some points, but I would be good to have navigation on sidelayout_nav allowing users to get a table view of all manifest (multiple sites per user). Maybe in the future will have basic ability to filter and drill down.
    • User can still drill down (read filter manifest by site) via the currently method that's similar to RCRAInfo (i.e., from home, go to site, go to site's e-Manifest module)
  2. Initial thoughts, handling all routing as a sub routing in the 'Manifest' feature directory. the Manifest component will be the top level component that will generally own manifest data and pass down data as props.

React Manifest View

Goal

This issue is intended to act as a tracker for progress made on using manifest via Haztrak

Frontend

Backend

  • Manifest ApiVIew (same class for all CRUD)
  • Require authentication
  • Search RCRAInfo Handler - endpoint

Docker compose

๐Ÿš€ Feature Request

add docker compose file to bring up client and server services, users with docker should basically be able to go from download, configure, and deploy locally in a couple commands.

Add 'haztrak_site' field to manifest model

Add new fields to model that corresponds to the site that owns the manifest

  • should be a foreign key that points to the site model
  • null not allowed

This will allow us to filter/display manifest by the site that owns it, instead of just looking for the handler. doesn't really make sense the way it is now, just haven't had time to fix.

considerations

  • Will require adjustment to serializer
  • We may need to adjust the services urls to reflect which site a manifest is being posted to
  • The SiteManifest view will need to be adjusted (primary reason for doing this)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.