innolitics / rdm Goto Github PK
View Code? Open in Web Editor NEWOur regulatory documentation manager. Streamlines 62304, 14971, and 510(k) documentation for software projects.
License: MIT License
Our regulatory documentation manager. Streamlines 62304, 14971, and 510(k) documentation for software projects.
License: MIT License
Our command rdm collect
grabs snippets from various files. Sometimes, e.g. in markdown files, it would be nice if the delimeters had the form
<!-- RDOC snippet_id -->
Snippet stuff goes here
<!-- ENDRDOC -->
This lets us delimit snippets without adding garbage to our markdown files.
Perhaps we can replace start_token
with start_regex
, where the regex has a capturing group for the snippet id. Thus, the HTML comment feature could use:
start_regex = '<!-- RDOC (.*) -->'
stop_regex = '<!-- ENDRDOC -->'
(or something similar).
It would also be nice to be able to grab snippets from python docstrings. E.g.:
class MyClass:
'''RDOC my_class
'''
# stuff goes here
Or, even more ideal, it would be nice if we could grab the python class snippets without needing RDOC
at all!
In the same way we provide a means for producing PDF documents, we should also provide a mechanism for producing word files.
A very basic question here: in some of the files in the document folder, there are some sort of tags, like this for instance:
{{ system.project_name }}
Is there a way to specify what this tag will be replaced with in the generated release version of the document?
Or do I manually replace the above string with the name of my project, everywhere it occurs?
This project is a great effort! Thanks a lot for starting it.
I'm giving it a try using the introductory guidelines.
However when I run rdm release
, I get the following error:
usage: rdm [-h] <command> ...
rdm: error: argument <command>: invalid choice: 'release' (choose from 'init', 'render', 'tex', 'pull', 'hooks')
I've tried this from within both the root of my repository and the regulatory
folder.
I get this error with any invalid command, e.g. rdm this-command-is-not-supported
, however release
is mentioned in the introductory guidelines, so it should be a valid command.
Per the 510(k) Format Tips document, item 3:
Provide a title and/or number for any diagrams, drawings, figures, illustrations, photos, charts, or tables that are used. Make sure citations in the text refer to them correctly.
We should figure out a way to support this sort of numbering.
data/abbreviations.yml
and data/definitions.yml
We want to make the variables in a document's YAML front matter available in templates.
It should work the same way Jekyll works:
https://jekyllrb.com/docs/front-matter/#custom-variables
Only, use "document" instead of "page".
Also, be sure to consider what would/should occur if there were a file named data/document.yml
. I propose that we simply disallow the use of the file data/document.yml
OR we merge it with the YAML front matter, having the front matter take precedence.
Once this change is made, we should update the init_files/documents/release_record.md
to set the release_id
with a document-level variable.
It would be nice if we could preserve the key order in YAML files that we dump.
This will probably involve writing a custom "Dumper" class for https://github.com/innolitics/rdm/blob/master/rdm/util.py#L16
Also, we will need to use ordered dicts in various places.
The templates have several TODO items. Many are necessary ones aimed at rdm users. They need to expand those to meet their specific situation. But some are really aimed rdm developers marking incomplete areas. The templates should be expanded to implement the latter. It would be useful, as a first step, to clearly mark which TODO is for whom.
Our GitHub-based project management backends require quite a few GitHub API hits. As a result, git pull
can be slow and it is possible to run into the GitHub API limit.
Ideally the solution would only pull down issues or pull requests which have changed since they were last pulled.
Hi, do you have plans to add templates for 62366-1? If not, would you accept contributions towards such templates?
I have not investigated in detail if 62366-1 is suited for your template system, but I am interested in starting that discussion at least.
When running rdm translate on xunit-style results, the flattened_gtest_results doesn't pick up req_ids (I'm not sure how it would?) and therefore when I go to make my documents, I get the following error:
File "./documents/test_record.md", line 36, in block "test_results" | {{ test_name }} | {{ unit_test_record[test_name].result }} | {{ unit_test_record[test_name].req_ids }} | {% if unit_test_record[test_name].note is defined %}{{ unit_test_record[test_name].note }}{% endif %} | jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'req_ids'
How would you plan to include req_ids from translate output? I use pytest for my python code and that provides the record_property fixture which can be used to inject into the XML report, which can then be picked up by translate.
Setup a public test repository with some data that exercises various corner cases.
Before implementing this, brainstorm with David + Yujan what corner cases we should cover.
Currently missing numerous references from all 6 62304 checklists
It would be nice to have a custom jinja tag for keeping up with definitions and abbreviations.
In particular, we typically use the same definitions in across different documents, but only want to include the relevant definitions in the documents where they are used.
To avoid duplication across different templates, it would be nice if we:
data/definitions.yml
) and abbreviations (data/abbreviations.yml
).The regulatory/data/system.yml
file contains two types of information:
repository
and software_requirements_location
.These two types of information should be split into two separate files.
regulatory/_config.yml
, which should contain configuration (this is analagous to the similarly named file in Jekyll)regulatory/data/global.yml
, which should contain global variables used in the default templates; these values should not have special semanticsQuestions:
rdm tex
command completely?Currently, rdm gap
only prints out the missing checklist items. It would be nice to provide an option that prints out the matching checklist items too. A few requirements of this option include:
This issue requires two reviews. First, a review of the design specification (including the example output format). Second, a review of the code construction and tests.
It is useful to include section numbers in markdown files. It also would be nice to have a way to pin the section numbers, so that they don't change over time, and/or you can skip over certain numbers if appropriate to simplify updating references to the document.
Update the RDM hooks so that, if there is no issue number, instead of discarding the commit message, we preserver it somehow. E.g., in the least we could print it out to the console so you can copy/paste it back in!
At the moment, it is frustrating if you forget an issue number and your commit message is lost as a result.
Hello,
Despite multiple efforts, I cannot generate the PDF files.
Make seems to stop after .md files generation.
Individual rdm tex
and latexmk
command works from command line.
My configuration is Windows 10 64 bits ; I tried Cygwin make and Gnuwin32 make with same result.
Any help is appreciated
We want to add a rdm subcommand that lets us audit a set of documents against a particular version of the standard.
This audit tool will take as its input a yaml file that includes the various required sections in a standard, as well as a set of markdown files to search. It will then look through the auditor notes to cross reference that all of the sections of the standard are addressed.
It should then output to standard output (the option to generate a Microsoft word file would also be good) that can be given to auditors as a starting place for the standard.
This will also facilitate performing a gap analysis whe up upgrading from one version of the standard to another.
Perhaps the subcommand should be called using this rdm gap -s 62304:2005 release/*.md
See checklist for IEC 62304 below:
Currently, the github-flavored-markdown tables, when converted to latex with pandoc, do not line wrap! This is problematic for definitions.
In particular, it would be nice if images posted in github issues would get pulled into the latex PDFs.
Images linked via URLs work in the markdown files already.
One possible (and probably necessary) solution will be to pull down images in URLs as part of the rdm tex
stage, and to place these images inside regulatory/tmp
.
A key part of RDM is connecting the commits to issues. The way we do this with Git and GitHub is to place Issue #201
in each commit message. Similar strategies would work with other version control systems and other issue trackers.
It can get annoying having to write Issue #201
into every commit you write. A common strategy to avoid this annoyance is to install githooks that place the issue number into your git commit message for you.
The way we have always done this in the past is to grab the issue number(s) off of the branch name.
If you are working on a branch for issue 201, you would name your branch: feature/201-brief-description
. Note that there may be multiple branches for one issue.
The githook would then use the branch name to add Issue #201
to the bottom of your commit messages.
We probably want support for multiple issues as well. E.g. if you are working on 201 and 207, your branch name would be feature/201-207-brief-name
. You would then want your commit messages to have Issues #201, #207
(or even Issue #201\nIssue #207
.
There are a number of side cases to consider.
git commit -m "My message"
would create a commit with the message:My message
Issue #201
If you type git commit
your editor should open up with the Issue #201
inserted in the commit message draft for you (you may need to change it; very occasionally you will have a commit on a branch that is actually not for Issue 201).
There are probably other cases I am not thinking about ... cherry picking---should be fine ... git commit -p
... should probably be fine. Anyway, try to think of other use cases.
We want to add an RDM subcommand that will install these githooks on a project. This sub command should work as follows:
rdm hooks
Should install the git hooks in the current git repository.
We may also want an option to install the hooks in a specified folder. Maybe something like:
rdm hooks specified/git/hooks/folder
The git hooks must be written with just bash, and should not use any other processes (except possibly git).
Some projects will have git hooks for other reasons! This complicates the process of installing them.
Ideally, we would have a way to install our scripts on top of existing scripts, but this may be complicated to setup.
It would be nice if we had some integration tests for this subcommand. Perhaps the integration test would:
One of our global template variables is a boolean called auditor_notes
. When this flag is true, we include notes with references back to the 62304 standard which will streamline internal and external auditing.
As a result, this syntax is very common
Some sentence{% if system.auditor_notes %} [62304:6.2.4]{% endif %}.
It would be nice if we could condense this down to:
Some sentence [[62304:6.2.4]].
Note the potentially tricky white-space handling.
It would also be nice if we could color these notes green in the final latex output.
I do not know if this is a feature, bug or neither, but when I run the make command the generated files are generated as ANSI format. This causes an error when trying to convert the generated document (e.g. release/510.md) to a word document using pandoc
Running MS Windows 10 Enterprise, Python 3.8
Add windows-latest
to the github actions tests workflow matrix and fix any bugs.
As of this writing, there is just one buy related to the tmp_editor
pytestt fixture.
I have run through the steps of the "Quick start", and added my own repo to the regulatory folder, by replacing repository: user/repository
(here) with my-sample-project
in the config.yml file
The regulatory folder in itself is not a git repo, so why does it need access to my source code?
Also when I make changes to the document templates, am I supposed to fork this project and push my changes to the forked repo, or should I create my own, private repo containing the regulatory folder ?
Pros of removing base templates:
Cons:
The checkbox lists:
- [ ] Item 1
- [ ] Item 2
are very convenient for verification records. It would be nice if the latex document was able to format these a bit better.
I get the following output:
See https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-linefor details.
There should be a space between command-line
and for
.
Auditors and regulators require that we keep a revision and approval history for our documents.
§820.40 Document controls.
Each manufacturer shall establish and maintain procedures to control all documents that are required by this part. The procedures shall provide for the following:
(a) Document approval and distribution. Each manufacturer shall designate an individual(s) to review for adequacy and approve prior to issuance all documents established to meet the requirements of this part. The approval, including the date and signature of the individual(s) approving the document, shall be documented. Documents established to meet the requirements of this part shall be available at all locations for which they are designated, used, or otherwise necessary, and all obsolete documents shall be promptly removed from all points of use or otherwise prevented from unintended use.
(b) Document changes. Changes to documents shall be reviewed and approved by an individual(s) in the same function or organization that performed the original review and approval, unless specifically designated otherwise. Approved changes shall be communicated to the appropriate personnel in a timely manner. Each manufacturer shall maintain records of changes to documents. Change records shall include a description of the change, identification of the affected documents, the signature of the approving individual(s), the approval date, and when the change becomes effective.
When using Microsoft word and excel, this process is usually handled by manually updating the revision history and revision number.
For documents stored in git, it probably makes sense to rely on git commits to track this.
Git commit messages, however, will not be accessible to non-developers. To circumvent this:
The history ideally will show line-by-line changes, who made the change, when, who approved the change, when, and when the change went into effect. To start, it probably makes sense to say that the change goes into affect once the document is approved... but it would be worth thinking about how this could be handled more flexibly in the future.
After this change, we should no longer need to manually edit the "revision number" in the YAML front-mater. That said, some people may still want to manually update this number and have it displayed in the revision history. Perhaps there is some way we can incorporate this with git tags?? It is a tricky issue.
Before implementation on this is completed, the architecture of the change should be mapped out and discussed with @johndgiese and @yujanshrestha .
The current setup only supports generating a single release document.
When I just run the Quick Start instructions from the README, I get the following error:
rdm pull config.yml > data/history.yml
Traceback (most recent call last):
File "/home/dheater/src/rdm-test1/venv/lib/python3.8/site-packages/rdm/main.py", line 21, in main
exit_code = cli(sys.argv[1:])
File "/home/dheater/src/rdm-test1/venv/lib/python3.8/site-packages/rdm/main.py", line 43, in cli
pull_from_project_manager(args.config)
File "/home/dheater/src/rdm-test1/venv/lib/python3.8/site-packages/rdm/pull.py", line 14, in pull_from_project_manager
BackendClass = load_class(config['project_management_backend'])
File "/home/dheater/src/rdm-test1/venv/lib/python3.8/site-packages/rdm/util.py", line 123, in load_class
module = import_module(module_name)
File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/dheater/src/rdm-test1/venv/lib/python3.8/site-packages/rdm/project_management/__init__.py", line 2, in <module>
from .github import GitHubIssueBackend, GitHubPullRequestBackend
File "/home/dheater/src/rdm-test1/venv/lib/python3.8/site-packages/rdm/project_management/github.py", line 6, in <module>
from github import Github
ModuleNotFoundError: No module named 'github'
Either the Quick Start instructions should use pip install rdm[svg,github]
to pull in Github support or maybe the default config.yml
shouldn't assume the GithubIssueBackend
(if that's possible?)
I am trying out your software on a dummy project
The documentation and the code standard of your project is awesome, however I have a couple of problems with the history.yml file generation. I am running Windows 10 Enterprise (x64),, with make installed with Chocolatey
Python3.8
I do the following steps (in Anaconda console)
rdm init
cd regulatory
git clone https://github.com/hmogensen/my-sample-project
make
Output:
(base) C:\Users\226077\Documents\sandbox\regulatory>make
rdm pull config.yml > data/history.yml
No access token is stored in the GH_API_TOKEN environment variable.
See https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-linefor details.
Defaulting to username / password for login.
hmogensen
GitHub password (will not echo to console):
Traceback (most recent call last):
File "c:\users\226077\anaconda3\lib\site-packages\rdm\main.py", line 21, in main
exit_code = cli(sys.argv[1:])
File "c:\users\226077\anaconda3\lib\site-packages\rdm\main.py", line 43, in cli
pull_from_project_manager(args.config)
File "c:\users\226077\anaconda3\lib\site-packages\rdm\pull.py", line 15, in pull_from_project_manager
pm_backend = BackendClass(config)
File "c:\users\226077\anaconda3\lib\site-packages\rdm\project_management\github.py", line 21, in init
self.github_repository = self.github_browser.get_repo(self.config['repository'])
File "c:\users\226077\anaconda3\lib\site-packages\github\MainClass.py", line 345, in get_repo
headers, data = self.__requester.requestJsonAndCheck(
File "c:\users\226077\anaconda3\lib\site-packages\github\Requester.py", line 315, in requestJsonAndCheck
return self.__check(
File "c:\users\226077\anaconda3\lib\site-packages\github\Requester.py", line 340, in __check
raise self.__createException(status, responseHeaders, output)
github.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest"}make: *** [Makefile:31: data/history.yml] Error 1
make: *** Deleting file 'data/history.yml'
If I rerun make
, only one file from the documents folder will be processed unless I comment out the commands @mkdir -p $(@D)
in the Makefile, since the script tries creating a new folder each time. After having performed this modification, running make yields the following output:
Traceback (most recent call last):
File "c:\users\226077\anaconda3\lib\site-packages\rdm\main.py", line 21, in main
exit_code = cli(sys.argv[1:])
File "c:\users\226077\anaconda3\lib\site-packages\rdm\main.py", line 36, in cli
render_template_to_file(config, args.template, context, sys.stdout)
File "c:\users\226077\anaconda3\lib\site-packages\rdm\render.py", line 38, in render_template_to_file
generator = generate_template_output(config, template_filename, context, loaders=loaders)
File "c:\users\226077\anaconda3\lib\site-packages\rdm\render.py", line 50, in generate_template_output
output_line_list = generate_template_output_lines(environment, template_filename, context)
File "c:\users\226077\anaconda3\lib\site-packages\rdm\render.py", line 62, in generate_template_output_lines
source_line_list = _generate_source_line_list(template, context)
File "c:\users\226077\anaconda3\lib\site-packages\rdm\render.py", line 91, in _generate_source_line_list
source = ''.join(generator)
File "c:\users\226077\anaconda3\lib\site-packages\jinja2\environment.py", line 1125, in generate
yield self.environment.handle_exception()
File "c:\users\226077\anaconda3\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "c:\users\226077\anaconda3\lib\site-packages\jinja2_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File ".\documents\release_history.md", line 31, in top-level template code
{% for cr in history.change_requests|rejectattr('is_problem_report') %}
File "c:\users\226077\anaconda3\lib\site-packages\jinja2\environment.py", line 471, in getattr
return getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'history' is undefined
When we include snippets inside documents, the header indent level may be different than what it is in the snippet source. It would be nice to add a filter so we could adjust this:
# My Document
{{ snippets.other_doc | mardown_indentation -1 }}
where the snippet is grabbed from here:
RDOC other_doc
### Header
Stuff here
ENDRDOC
See http://jinja.pocoo.org/docs/2.10/api/#custom-filters
This issue is just a placeholder, so discuss with David + Yujan before implementing.
We need to produce a nice looking risk management matrix etc. from this document:
https://github.com/innolitics/rdm/blob/master/rdm/init_files/data/risk.yml
This is a big issue and will require some work to setup in a sufficiently flexible manner. It may require its own "backend" system.
We should also discuss whether it is possible to automatically vs manually tie tests to their requirements.
Finally, it would be great if the document(s) generated from the traceability can include links back to the github code!
It would be helpful to have categories in the requirements specification. The the requirements could be grouped by category with a header as the category name. For it is helpful to have a category for each software module and then have requirements for that module. Or to separate function and performance.
We will slowly add checklists for a few different FDA Guidance documents. Requirements in guidance documents will likely be less clearly defined than in ISO standards.
It may also be useful to extend the checklist format a bit so we can keep the documents simple. In particular, it may be nice to include lines like:
FDASG:4.2==62304:5.1.3 Optional comment goes here
to indicate that meeting this part of 62304 also meets the related guidance requirement.
It would be nice that rdm handles gitlab in the same way it does with github.
Use the GitHub API to construct the requirements.yml
file and problem-reports.yml
files.
Both of these files should have the same format:
1: "description of the requirement or problem report in GitHub flavored markdown"
3: "description of the requirement or problem report in GitHub flavored markdown"
4: "description of the requirement or problem report in GitHub flavored markdown"
We should generate both yaml files using a new Makefile rule and a new rdm subcommand
.
This will require:
.yml
fileWe should use an existing GitHub API library, e.g. http://pygithub.readthedocs.io/en/latest/introduction.html
When executing rdm pull
without access token the user is prompted for github username and password. This interaction works fine except that the resulting history.yml
file starts with:
GitHub username: change_requests:
instead of the expected:
change_requests:
This causes later commands to fail with errors like:
yaml.scanner.ScannerError: mapping values are not allowed here in "<unicode string>", line 1, column 33: GitHub username: change_requests:
The work around, until this is fixed, is to edit out the leading GitHub username:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.