18f/micropurchase

18F's micro-purchase threshold experiment management app.

Micro-purchase

This is a web application used to manage the bidding process for 18F's micro-purchase threshold experiment. The platform will allow vendors to bid on open opportunities with 18F, track their bids, and learn of the winning bidder. So long as vendors are registered on SAM.gov and have GitHub accounts, they will be able to view open opportunities and bid on them.

With this application, a vendor will be able to view the full list of open micro-purchasing opportunities, access bid histories, and place bids on services requested by 18F. All bids will start under $3,500 and each project will specify the desired product and method of delivery.

This is a Ruby/Rails application using ActiveRecord and PostgreSQL. This repo contains the front end of a web app that integrates GitHub and SAM.gov. For more information on setting up the back end of the web app, see below.

• Staging: https://micropurchase-staging.18f.gov/
• Production: https://micropurchase.18f.gov/
• API docs: https://micropurchase.18f.gov/api
• Problem statement
• Roadmap
• Leads tracker
• Backlog

Documentation

Local Development

See the CONTRIBUTING.md for a full run-down of how to contribute to this application.

Database Schema

We are keeping a version-controlled Entity Relationship Diagram (ERD) located indocs/erd.pdf. Any new change to the database schema must include an update to this diagram. You can automatically update the diagram by running (follow the local development instructions below if you don't have the app setup locally):

[docker-compose run web] bundle exec erd

Updating the ERD requires Graphiz. Installation instructions are here.

Coverage and CodeClimate

Because this application uses two different test suites (RSpec and Cucumber), it has a more complicated setup for measuring coverage and reporting it to CodeClimate. By default, CodeClimate only will use the coverage statistics from RSpec, meaning you will see a drop in coverage for controllers tested more thoroughly by Cucumber. The solution involves a few parts:

1. The .simplecov file in the root specifies SimpleCov configuration shared by both RSpec and Cucumber. This switches the coverage gem to use for CI vs. local operation.
2. The codeclimate_batch CLI merges the coverage reports from each suite before reporting to CodeClimate. This process includes sending all configs to the cc-amend service. To use the them, you must also make sure your CI calls bundler with --binstubs to install gem binaries locally.
3. The codeclimate_batch gem will only run on a CI server and you must also define an environment variable CODECLIMATE_REPO_TOKEN with the value of the repo token provided by CodeClimate for it to work.
4. The codeclimate_batch gem will also only run on the DEFAULT_BRANCH specified in the .simplecov file. If you change your CodeClimate to use a branch other than develop, you must change the value in .simplecov

If everything is working correctly, you should see the following text at the bottom of CI builds of your develop branch:

$ ./bin/codeclimate-batch --groups 2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 23656  100    41  100 23615    202   114k --:--:-- --:--:-- --:--:--  118k
sent 2 reports for 18F-micropurchase-1248
Code climate: 0.21s to send 2 reports

Security Scans

This repository uses two tools to provide a total of three types of automated security checks:

• Brakeman provides static code analysis.
• Hakiri is used to ensure the Rails/Ruby versions contain no known CVEs.
• Hakiri is used to ensure the gems declared in the Gemfile contain no known CVEs.

All security scans are built into the test suite. bundle exec rake spec will run them. To run the security scans ad hoc:

Brakeman:

bundle exec brakeman

Hakiri for Ruby/Rails versions:

bundle exec hakiri system:scan -m hakiri_manifest.json

Hakiri for Gemfile dependency versions:

bundle exec hakiri gemfile:scan

Ignored Brakeman warnings

Sometimes Brakeman will report a false positive. In cases like these, the warnings will be ignored. Ignored warnings are declared in config/brakeman.ignore. This file contains a machine-readable list of all ignored warnings. Any ignored warning will contain a note explaining (or linking to an explanation of) why the warning is ignored.

Public domain

This project is in the worldwide public domain. As stated in CONTRIBUTING:

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

micropurchase's Issues

As a bidder, I want to be able to make multiple bids

Given I am an authenticated and verified bidder
And I have entered a bid
And someone else has entered a lower bid
Then I should be allowed to enter another bid to stay in the competetion

Add feature specs to exercise all views and articulate user flows

Currently there are controller specs that test database changes and which view is rendered. There have been easy refactoring generated bugs getting through because the views are not being exercised in the tests yet. Also, the typical user story flows are not being captured in the unit tests.

This chore aims to capture user flows and make sure all the views get exercised in the test suite. More than likely controller specs will be eliminated in this process.

Add licensing, contribution files

Please add the standard LICENSING and CONTRIBUTION files.

As an 18F developer, I want Rails ...

As an 18F developer
I "want" Rails
So that I can develop faster, better.

As a site visitor, I want to see a favicon ...

As a site visitor
I want to see a favicon
So that I can quickly understand what site I'm visiting by looking at the browser tab.

As a bidder, I want to store my sam and duns number in the app

Given I am prompted to authenticate myself via github
When I return to the app
Then I should see a form demanding my duns and sam id
And I must enter those details in order for my profile to be saved and to be logged in
And my bidding status will not be approved

Test ticket from Code Climate

As an admin, I want to create new issue/auctions

Given I am logged in as an admin
And I go to the admin/issues/new page
Then I should see a form for entering new auctions into the app
When I complete the form
Then I will have created a new auction with the details provided

Given I am a non-admin signed in user
When I try to go to the new issues page
Then I will be redirected to home

Add tests

As an admin, I want to sign in without being asked to provide a DUNS or SAM number,

As an admin,
I want to sign in without being asked to provide a DUNS or SAM number,
So that I don't enter un-needed information.

As a bidder, I want a clear indication of whether my bid on an issue is the winning bid

Given I am an authenticated and approved bidder
And I have posted a bid
And I am the lowest bid
Then I want to see on the issue detail page a clear indication that I am the current winner

As an admin, I want to edit existing auctions

Currently you can create auctions, but no edit them.

Change the bidding increments

As a bidder, I want to know what the minimum bid increment is, so that I can bid appropriately.

Fix possible unprotected redirect

  1) Security (via Brakeman static code analysis) The site has zero Brakeman security warnings
     Failure/Error: expect(@brakeman_warnings.length).to eq(0), "Expected 0 security warnings, got: \n #{@brakeman_warnings.map(&:to_s)}."
       Expected 0 security warnings, got: 
        ["(High) Redirect - Possible unprotected redirect near line 3 in /Users/alandelevie/micropurchase/app/controllers/authentications_controller.rb: redirect_to(Authenticator.new(request.env[\"omniauth.auth\"], session).perform)"].
     # ./spec/features/security_spec.rb:14:in `block (2 levels) in <top (required)>'

More info: http://brakemanscanner.org/docs/warning_types/redirect/

Add a Procfile to config for production

Read more: https://docs.cloudfoundry.org/buildpacks/ruby/ruby-prod-server.html

As a site visitor, I should be able to see all the open issues

Given I have not signed in
Then I should be able to see all the open issues

As a site visitor, I want a "Login" link in the header.

Given that I'm not registered / have an account,
I see a login link in the header,
which takes me to the registration / "Getting Started" page,
so that I can authorize with GitHub.

Header at home screen:

Registration / Getting Started page:

Ensure relevant documentation is up-to-date

See https://pages.18f.gov/before-you-ship/ato/

Please re-open this issue if any relevant documentation falls out of date.

As an 18F developer I want a reliable CI/CD pipeline.

As an 18F developer
I want a reliable CI/CD pipeline
So that ...

As an admin, I want to manage data...

Given I'm authenticated as an admin (could be anyone in consulting group),
I can manage any models in the database via CRUD operations.

As an admin I want to receive email

As an admin
I want to be able to receive email from users
So that I can respond to their needs.

Don't push directly to master (use pull requests, do code reviews)

Rather than pushing directly to master, can we use pull-requests? I'd love to see (1) pull requests for each change and (2) code reviews

Ensure that the pull request is merged by someone other than the original author.

As an authorized user, I want a "Logout" link in the header.

Given that I am authorized with GitHub,
I want to see my avatar in the Header,
and have a link to "Logout".

Rename repo (and other references) to "micropurchase"

MPT3500 was a working title, but we want this whole shebang to be more accessible.

Avoid split purchases

Not for the upcoming sprint, but these are proposed scenarios for the future...

Feature: Avoid split purchases
  The micro-purchase platform should automatically flag situations that pose a risk of split purchases
  The primary concerns are related to similarly scoped requirements or multiple requirements in a short time frame
  To help address, require a review and/or require a heightened "power user" level

Scenario: Multiple requirements within a month get flagged for review
  Given a requirement R for a project repository P
    And R was opened within the past 28 days
  When there is a request to open a new requirement N for P
  Then flag N for review by power user

Scenario: Multiple requirements by non-power-user within 48 hours get flagged for review
  Given a requester Q is not a power user
    And Q has submitted a requirement within the past 48 hours
  When Q requests to open a new requirement N
  Then flag N for review by power user

Obviously, this implies not just the new functionality, but will require some notion of "role" for a given administrative user.

As a admin, I want to approve bidders so that they can start bidding on auctions/issues

Given I am a logged in admin
And I visit the adming/bidders page
I should see a list of bidders with information
And each bidder should have an indication of whether they have been approved
When I click to approve a bidder
Then I should see that they have been approved
When I click to remove approval from the bidder
Then I should see that they are no longer approved

Placeholder for micro-purchase

[Placeholder for micro-purchase]

As a DevOpser, I want to know the architecture and data flows of the application

As a DevOpser
I want to know the architecture and data flows of the application
So that I can mitigate problems during the ATO process.

See https://pages.18f.gov/before-you-ship/ato/.

• What does it do?
• How does it move information around?
• What does it accomplish by doing it?

The documentation should be in the README.

Select security controls

See https://pages.18f.gov/before-you-ship/ato/.

Update UI/UX based on mockups

Current mockups here – post a comment if you have feedback on them:

As a bidder, I want to be able to make a single bid

Given I am an authenticated
I should be allowed to enter a bid

As a site visitor, I should be prompted to sign in when trying to bid on an issue

Given I have not signed in
And I visit the detail page of an issue
Then I should see a button prompting me to sign in to bid
And when I click it, I should be redirected to github to authenticate
And when I return I should be prompted for my duns and sam id
And when I complete that information, I will finally be redirected to the detail page containing a bid form

Data flow documentation

Admin data flow

An admin is authenticated via GitHub-provided OAuth (using the omniauth-github gem) and authorized via a whitelist of GitHub user IDs (see GitHub documentation for instructions on obtaining a canonical ID for a GitHub user.) The whitelist is public, version-controlled, and located at config/admins.yml. Only persons who are permitted to have read and write access to all data are included on the whitelist.

Successful authentication results in the creation of a session which stores a unique identifier of the user.
Logging out occurs after a successful GET request to /logout. Logging out results in the destruction of the previously-mentioned session.

---

#### End-User Registration Data Flow

The registration data flow describes the process by which both admin and non-admin users register to use the Micropurchase website.

In screen 1, the user is logged out, and clicks on a button taking them to /login. /login is a public-facing page that includes information about registration as well as any terms that must be agreed to before proceeding.

On screen 2, when the user clicks "Authorize with GitHub", they proceed through the GitHub OAuth flow. After granting the Micropurchase OAuth application permission to access public information from their GitHub account (on screen 3), the user is redirected back to the Micropurchase website (screen 4).

If the GitHub ID does not exist at this time in the Micropurchase database, a new user record is created. Screen 4 contains a form where for non-admin users to enter their DUNS number. Upon submission of that form, the DUNS number is saved as part of that user record. A DUNS number is public information. At this stage, there may be validation of the DUNS number. Such validation may include a request to the SAM.gov API to ensure authenticity of the DUNS number and/or to retrieve public information associated with that DUNS number from SAM.gov. This public information may be presented back to the user for confirmation, and this information may be stored in the Micropurchase database as part of the user record.

Once a user has submitted their DUNS number, they may place bids, using the bid data flow described above.

Successful authentication results in the creation of a session which stores a unique identifier of the user.
Logging out occurs after a successful GET request to /logout. Logging out results in the destruction of the previously-mentioned session.

---

#### Bid data flow

The bid data flow describes the creation of bid records in the database. This diagram assumes a user is already registered and logged in (e.g. authenticated). In screen 1, the user clicks on a an individual auction listing and is taken to screen 2. In screen 2, the user types a bid amount into a form, and clicks submit. The same either page re-renders or the user is redirected to another page. Meanwhile, if the bid amount passes validation checks on the server, a new bid is saved in the database.

The bid list screen displays a list of bids for a given auction. All bid data is considered public and not procurement sensitive. At the discretion of the Micropurchase team, some data (such as identity of the bidder) may be redacted while an auction is currently running. However, all users will be given notice and must consent to the publishing of all bid data (and potentially other usage data) before registering and before bidding. Terms will be consistent and made clear to users. Those terms may change from auction to auction, however, terms will be consistent within each auction. All bidding data will be released when an auction ends.

---

Enable analytics (DAP)

Max bid price for new bids

On /make-bid, each bid must be less than the previous bid.

As a developer, I would like to see CodeClimate post commit hooks, so that I know what effect my work has on the overall code quality

Adding the commit hook needs to be done by the organization owner. Currently I have the badge setup via my own login. I would really like to get this in our commit process ... and even better I would like to see code coverage too.

http://docs.codeclimate.com/article/222-how-do-i-install-code-climates-github-service-hook
http://docs.codeclimate.com/article/219-setting-up-test-coverage

Expand the README

Flush out the README so that it's clear what's going on here.

What does this software do?
Why is it needed?
What's the micro-purchase experiment?

If one wants to run a local copy of this, how does one do it?

As a site admin I want CRUD access to each model

As a site admin
I want CRUD access to each model
So that I can moderate Auctions, Bids, and Users

[A little higher fidelity stories:]

As a site admin
I want to create/edit Auctions from the web UI
So that I can easily add/edit Auctions and not need SSH access to the server

As a site admin
I want to monitor bids from the web UI
So that I can better understand the auctions (e.g. who is bidding, what prices are the bids, etc)

As a site admin
I want to view a list of users
So that I can know who the users are, and
So that I can begin to verify their DUNS numbers

As a site admin
I want to be able to delete abusive users
So that I can maintain the integrity of the Auctions

As a site visitor, I want clear instructions ...

As a site visitor
I want clear registration and bidding instructions
So that I know if bidding is right for me

As open data, I want to be liberated through an API.

• association between GitHub user/org I.D. and DUNS/SAME
• information about all auctions, past and present
• information about all bids, past and present

Implementing the controls

See https://pages.18f.gov/before-you-ship/ato/

Selected controls will be visible here: #62

Use U.S. Web Design Standards gem

See gem repo for details.

Conducting a preliminary code review

See https://pages.18f.gov/before-you-ship/ato/

Add a `manifest.yml`

See e-Manifest's: https://github.com/18F/e-manifest/blob/master/manifest.yml

As a site visitor, I should be able to see the detail view on any issue

Given I am not signed in
Then I should be able to click on the detail view for any issue
And I should see details about the current bid

As a site visitor, I want to understand the scope and experimental nature of the project.

Link to blog post?
Clear feedback or issue tracking system

As a bidder, I want to know that I am logged in by seeing my github info in the header

Given I have signed in as a bidder with github
Then I should see my name and github image in the header

Ensure any visible security issues are fixed

See https://pages.18f.gov/before-you-ship/ato/

If any security issues are identified, please reference this issue and re-open it if the issue is close.

Drawing a top level diagram the system architecture

See https://pages.18f.gov/before-you-ship/ato/

As a user, I want real data to populate the views.

None of the views use real data at the moment. We'll use ERB for this.