18f/atf-eregs

Container and styles for an ATF eRegs instance

Moving to @atfweb - email the team at [email protected] with any questions.

---

ATF eRegulations

This repository contains code necessary to run a Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) instance of eRegulations (a regulation parser, API, and viewer). Live version: https://atf-eregs.app.cloud.gov/

This code glues together general-purpose/non-agency-specific eRegulations libraries (which are not in this repository) with ATF-specific styles, templates, and plugins (which are in this repository).

Status

Documentation and contributing

See the eRegulations overview for context about eRegulations, which is a multi-agency project.

To learn how to set up ATF eRegulations (locally or in production) and customize it/develop for it, see the documentation hosted on Read the Docs.

If you're interested in contributing to ATF eRegulations, see the contributing guidelines.

Public domain

This project is in the worldwide public domain. As stated in CONTRIBUTING:

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

atf-eregs's Issues

CFPB update made headers super bold

Compare

to

We don't have Avenir Next, which I think is the underlying problem.

555 - Footnotes

27 CFR 555 has footnote tags. We need to encode these into the tree/layers and modify -site to display them.

Markdown has a footnote definition; might make sense to adopt that.

EXTRACT display

Should word wrap

Agency-specific stylesheets

We need to pull in regulations-site's less files, but combine them with ATF-specific color values

Add about.yaml

See https://github.com/18F/about_yml

Explicit label in AMDPAR and MARKERLESS

https://www.federalregister.gov/articles/2009/01/14/E9-578/commerce-in-explosives-amended-definition-of-propellant-actuated-device-2004r-3p#p-amd-3 and https://www.federalregister.gov/articles/2014/08/11/2014-18842/technical-amendments-to-regulations#p-amd-51 refer to specific, markerless paragraphs. Unfortunately, using the [label:555-11-p65] syntax does not work as the paragraph is parsed as 555-11-p1.

regparser.notice.changes has some functions around "candidates" and misparsed changes which would be good to leverage.

Link color is too dark

Particularly when the copy is black

ATF Colors

Should match atf.gov

Missing icons in navigation sidebar on Windows

Some of our ATF users can't see the icons at the top of the navigation sidebar, as shown in this image.

Incorrect definition

“I declare under the penalties of perjury that this—(insert type of document, such as, statement, application, request, certificate), including the documents submitted in support thereof, has been examined by me and, to the best of my knowledge and belief, is true, correct, and complete.”

is being picked up as a definition. Probably worthwhile to both limit the length of a definition to prevent this

ATF Fonts

Should match atf.gov

555 - Extracts

555 has a few sections (particularly tables) which live inside EXTRACT tags. At the moment, we're converting EXTRACT tags into fenced markdown and treating subparagraphs as lines of text.

We probably need to create a new type of Node to represent the root of an EXTRACT, allowing it to have subnodes to represent sub paragraphs, children, etc. We'll need to modify -site to display accordingly.

Remove references to `eregsip`

[14/Sep/2015 18:45:55]"GET /static/eregsip/font/1e9892c0-6927-4412-9874-1b82801ba47a.woff HTTP/1.1" 404 1775
[14/Sep/2015 18:45:55]"GET /static/eregsip/font/91b50bbb-9aa1-4d54-9159-ec6f19d14a7c.woff HTTP/1.1" 404 1775
[14/Sep/2015 18:45:56]"GET /static/eregsip/favicon.ico HTTP/1.1" 404 1670

Pull in latest CFPB code into -site

Move auth info into a user provided service

Right now, the write api is protected via http auth, which is configured via env vars. Unfortunately, this means that a cloud foundry manifest file can't be complete. The approach @jmcarp's recommended sounds like a good solution on the surface.

Dive in and make that service if it is appropriate.

atf-eregs needs to build the frontend

Before, we were relying on pre-compiled CSS/JS in regulations-site. Now that we've pulled in CFPB's latest, those source files etc. are gone.

Ideally, we wouldn't go back to checking those files into the repo. Look into cloud foundry build scripts to investigate where/when building can occur. Relates to #3

Stylesheet updates

• Embed fonts rather than using an external reference
• Go through the less files in #34 and pull out anything that could apply in regulations-site (to minimize the overrides). Preferably we'd only be updating mixins.less and fonts.less

Hard reloading a section with alpha doesn't work

Ajax works a-okay, but hard loading does not

479 landing page

Include 18f in the about page

eRegulations makes regulations easier to read and navigate. It clarifies regulations by bringing related information and regulatory history to the forefront. It is a work in progress by the Consumer Financial Protection Bureau, and is a public domain work of the United States Government.

Because we helped, too!

555 Verification

• 03-1657 - Ignored; 03-6573 is the beginning of time
• 03-6573 - Annual edition, 2003 #101
• 03-23093 - Annual edition, 2004 18F/fr-notices#8
• 05-10618 - Annual edition, 2006 (though see #120)
• 06-6862 - Annual edition, 2007 #121
• E8-23178 - Notice must be parsed
• E9-578 - Annual edition, 2009
• 2010-891 - Annual edition, 2010
• 2014-18842 - Annual edition, 2015
• 2015-25190 - Notice must be parsed 18F/fr-notices#10

Analytics

Preferably using DAP

Subject groups in diff view

For the time being, subject groups should just be removed from the TOC in diff mode

Header issue: 32a

When first clicking to section 32a:

Then, after scrolling:

New Relic Setup

Not a must have, but would be useful

API Cache Controls

The data in the API basically never changes. Once we're behind api.data.gov, it'd be nice to rely on their caching, which means we need to set the appropriate cache control headers.

Use api.data.gov to front the API

• Add an api.data.gov endpoint
• Limit non-local traffic to api.data.gov end points
• Depend on api.data.gov for write limitations

compile_frontend touchups

• Add frontend_build to gitignore so that it's not pushed with cf (the dir is almost a quarter a gig)
• Add settings for finding the new static files
• I had to move static/* into static/regulations/* manually

Back button to landing page does not work

In a modern browser (i.e. one w/ push state)

1. Go to the root (universal landing page)
2. Click a regulation -- you'll get to the regulation's landing page
3. Click a section in the TOC
4. Click the back button -- the url changes to the landing page but the page content does not

Fix header for subject groups

Replace images w/ blue versions

There are a few static images, particularly in the "About" page, which need to be re-create or modified to be blue rather than green

EXTRACT Sandwich

In one annual edition, we have:

<EXTRACT>
    <TAG1>....</TAG1>
    <GPOTABLE>...</GPOTABLE>
    <TAG2>...</TAG1>
</EXTRACT>

and in the next:

<EXTRACT>
    <TAG1>....</TAG1>
</EXTRACT>
<GPOTABLE>...</GPOTABLE>
<EXTRACT>
    <TAG2>...</TAG1>
</EXTRACT>

It's safe to say that the table is supposed to be inside the extract, so let's add a preprocessing step to account for that kind of error.

Host both -core and -site from this repository

Right now, we're deploying two instances. Instead, move all of the atf-specific code (including cloud foundry setup) into this repo.

Fix header for markerless paragraphs

32a in wrong place in diff timeline

at /diff/479-172/03-1657/2014-18842?from_version=2014-18842

"About eRegulations" is black on dark

Fix -site and -core setup.py

Right now, we need to keep the requirements.txt file in this repository in sync with both -site and -core. That's silly -- we should be using proper dependency management via setup.py

API Documentation

It'd be nice to have a swagger-esque interface for the API

Prep settings for final deployment

• Django 1.8
• DEBUG = False + templates
• ALLOWED_HOSTS
• Credential-protect the write API

Internal citations for sections with letters

For example, 25 CFR §479.32(b) refers to § 479.32a, but the current internal citation only matches 479.32

555: Collapsed markers in markerless

The definitions in part 555 often have a collapsed (a) marker, which we're not catching. The paragraphs are then parsed as, e.g.

MARKERLESS
    b

or worse:

MARKERLESS
        1
        2
    b

which cause parse errors

Deleting a section doesn't work

Found in 555's 05-10618

The code for deletion does not correctly find a parent for sections. This will be a risky change as it's possible existing notices relied on this bug.

Fix search results for markerless paragraphs

About page style love

I think the larger font caused issues:
ATF:

vs. CFPB:

About page text

Remove references to section-by-section analyses and inline interpretations as ATF does not have these

Footer text

The full_footer template includes some dummy text which should be removed.

555 - Headers

27 CFR 555 has a few stray HD tags which we don't account for at all. It might make sense to convert these into Nodes with no text and only a title and/or to combine them with tables where appropriate. In any event, we need to store and display these headers

27 CFR 646 Annual Edition Verification

• 03-1657 - Annual edition 2003
• 2014-18842 - Annual edition 2015

Notice JSON needs more attributes

The write_to command is missing a few key fields when it writes notices. fr_url and publication_date are required. Further, the Section-by-section analysis is missing at least fr_volume, regulation_id_numbers, action and document_number.

Add back those pieces of meta data when parsing the SxS information. This structure is then sent to the API as a notice.

27 CFR 479 Annual Edition Verification

Make sure that all discrepancies between the annual-edition building scripts and the live data can be accounted for.

• 03-1657 - Annual 2003
• E8-23178 - Annual 2009
• 2014-02580 - Must be parsed
• 2014-18842 - Annual 2015