Giter Site home page Giter Site logo

ministryofjustice / prison-api Goto Github PK

View Code? Open in Web Editor NEW
6.0 40.0 6.0 84.11 MB

API for Nomis DB used by DPS applications and other apis and services

Home Page: https://api.prison.service.justice.gov.uk/swagger-ui.html

License: MIT License

Dockerfile 0.01% PLSQL 12.24% Shell 0.01% Java 55.60% Gherkin 1.66% Kotlin 30.48%
prison hmpps

prison-api's Introduction

Prison API server

CircleCI API docs

How do I get set up?

  • If using Intellij, install Lombok

  • The mobile API Server uses the configuration file in YAML format. There is at least one initial configuration packaged in the jar on

    prison-api/src/main/resources/application.yaml

    Let's say you want to create a specific configuration to the DEV profile: just copy a file to your configurations.

    prison-api/src/main/configs/application-dev.yml

  • Dependencies To minimise dependency management we use Spring Boot because it gives us a broad set of frameworks with the right version avoiding library conflicts.

  • Build command

    gradlew clean build

  • Database configuration To connect the application to a database, change the configurations in your configuration profile file, for example in mobile-dev.yml.

How to run the application?

  1. Run uk.gov.justice.hmpps.prison.PrisonApiServer using spring boot profile nomis-hsqldb

  2. Using Gradle directly via command-line (optionally setting the spring profile variable to select the in-memory database)

Windows

set JAVA_OPTS=-Dspring.profiles.active=nomis-hsqldb
gradlew bootRun

Mac

./gradlew bootRun --args='--spring.profiles.active=nomis-hsqldb'

Running the application against dev (t3) database

Steps are:

  1. Create a tunnel to the database
ssh -f -D 1086 -N -Snone hmppgw1 -L1521:t3nomis-b.test.nomis.service.justice.gov.uk:1521

For the tunnel to work you will need host information for hmppgw1 in your ~/.ssh/config. Please see confluence for more information. 2. Start Prison API with the following VM options

-Doracle.jdbc.J2EE13Compliant=true -Xmx1024m -Xms1024m

and with profiles

nomis

and the following properties

spring.datasource.url=jdbc:oracle:thin:@localhost:1521/NOMIS_TAF
spring.replica.datasource.url=jdbc:oracle:thin:@localhost:1521/NOMIS_TAF
spring.datasource.username=<your t3 user>
spring.datasource.password=<your t3 password>
spring.replica.datasource.username=<your t3 user>
spring.replica.datasource.password=<your t3 password>
spring.security.oauth2.resourceserver.jwt.jwk-set-uri=https://sign-in-dev.hmpps.service.justice.gov.uk/auth/.well-known/jwks.json

Running

http http://localhost:8080/health

will then test that the application is up and running successfully.

In order to then make calls to Prison API you will need to call HMPPS Auth first in dev to get a token, in the same way as calling any Prison API endpoints in dev.

Authenticating with JWT when running locally

When running locally with the profile dev JWTs are verified against http://localhost:9090/auth/.well-known/jwks.json. When deployed to t3, t2, preprod or prod, JWTs are verified against the public key hosted by the auth server at https:///auth/.well-known/jwks.json.

Coding Standards

There are a few different styles in this project due to historical reasons and the accumulation of technical debt.

The preferred patterns going forward are:

  • Use Spring Rest Controllers for APIs
  • Use thin controllers and apply business logic in an associated service
  • Use SpringJPA for database access
  • Use WebTestClient for integration testing (e.g. extend ResourceTest, do not use Cucumber)
  • Use ControllerAdvice for error handling
  • Do not create interfaces with single implementations

There are some recent examples around PrisonStatusController and OffenderImageRepository and AgencyResourceImplIntTest.

Health

  • /health/ping: will respond {"status":"UP"} to all requests. This should be used by dependent systems to check connectivity to prison api, rather than calling the /health endpoint.
  • /health: provides information about the application health and its dependencies. This should only be used by prison-api health monitoring (e.g. pager duty) and not other systems who wish to find out the state of prison api
  • /info: provides information about the version of deployed application.

Querying a local HSQLDB Database

The feature tests and repository integration tests create and populate a local in-memory HSQLDB called nomis-db. This is by definition transient but it can be very useful to persist this database temporarily in order to execute test queries, examine the data and check your repository queries.

To do this:

  • Alter the file application-nomis-hsqldb.yml to use a file-based url. In this example I created a directory ~/dbs under my home directory first.

spring: datasource: url: jdbc:hsqldb:file:~/dbs/nomis-db;sql.syntax_ora=true;get_column_name=false;shutdown=false;sql.nulls_first=false;sql.nulls_order=false username: sa password: changeme

  • Add a temporary password to the spring datasource (by default its blank) which then does not allow a client connection via the IntelliJ database tools.

  • Run one of the repository integration tests in IntelliJ to create and populate the DB and let it finish and exit. As its now file-based the database remains intact.

  • Within IntelliJ, select the Database tab and add a datasource with the following properties:

DriverType: HSQLDB Connection type: URL only URL: jdbc:hsqldb:file:/<your home dir>/dbs/nomis-db;sql.syntax_ora=true;get_column_name=false;shutdown=false;sql.nulls_first=false;sql.nulls_order=false;hsqldb.lock_file=false User: sa Password: changme

  • Connect, open an SQL editor or browse the tables with the test data present.

  • Important: Close the connection and remove the database files before attempting to re-run any integration or feature tests.

    $ cd ~/dbs $ rm -rf nomis-db*

  • Don't commit these changes!

Environment Variables

Below are the key environment variables that can be set per service:-

SPRING_PROFILES_ACTIVE=nomis
SPRING_DATASOURCE_URL=<jdbc datasource url>
SPRING_DATASOURCE_PASSWORD=<db password>
  1. In a Docker container

Build Docker image and run

./gradlew clean assemble

docker build -t quay.io/hmpps/prison-api .

docker run -d -p 8888:8080             \
       -h prison-api                   \
       --name=prison-api               \
    quay.io/hmpps/prison-api:latest

Using Docker compose

docker-compose up -d

Running Feature Tests

All feature tests can be run from uk.gov.justice.hmpps.prison.executablespecification.AllFeatureTest

Tests can be run individually by adding an e.g. @wip tag to the top of the feature file, or to an individual feature scenario, and then adding the environment variable cucumber.filter.tags=@wip to the executable arguments in 'Edit Confgurations'.

To run all feature tests in IntelliJ, avoiding any known @wip or @broken tests, set cucumber.filter.tags=not(@wip or @broken)

Authorize with Swagger UI

  • Without correct authorisation calling endpoints will return http 401 - Unauthorized
  • Obtain a correct JWT Token from 'Auth' service
  • Go to http://<host:port>/swagger-ui/
  • Click on top right button 'Authorize'
  • In the 'Authorize' dialog enter
    • Bearer <Your Token>
    • Like so Bearer eyJhbGciOiJ...OsgGjHBuA
  • Then click 'Authorize' and close dialog
  • The endpoint calls should pass security and allowed through now.
  • When the token expires logout of the 'Authorize' and enter a fresh token.

Alerts

Inactivity Alert

There is an alert in Application Insights called prison-api - Inactivity alert. It fires if prison-api hasn't received any successful requests in the last 10 minutes.

If the alert fires then look for any recent releases of prison-api that may have introduced a problem. If not then check in the nomisapi-prod Kubernetes cluster looking for any errors - look at events, ingresses, certificates etc. Finally it could be some kind of network issue - check for problems in Azure AKS.

Offender images

  • Offender images can be set using the endpoint PUT /api/images/offenders/{offenderNo}
  • This requires a token with the role ROLE_IMAGE_UPLOAD
  • The service scales the image into two, one thumbnail and one full-sized.
  • The service assumes a 4:3 aspect ratio, and also looks ok for 16:9, but square images will distort slightly.
  • Compact digital cameras and phones use 4:3 generally, and others 16:9.
  • Max size is 1048576 bytes (can be configured)
  • If no booking is found for the offender this will fail with a 404 and appropriate message.
  • The previously active image will be set to inactive.

Uploading an image

  # PUT an image for an offender
  curl -X POST "https://<URL to prison API host>/api/images/offenders/{prisonerNumber}" \
     -H "Authorization: $AUTH_TOKEN" \
     -F file="@./image.jpg" | jq

prison-api's People

Contributors

andrewrlee avatar andymarke avatar avenak avatar bell-pa avatar brightonsbox avatar carloveo-moj avatar davidmiddletonmoj avatar ed209uk avatar garethmdavies avatar hyderkhanmoj avatar jamesak-oncloud9 avatar jamiedraperuk avatar karenmillermoj avatar lbennett-moj avatar ldlharper avatar matt-crowson-moj avatar mattonfoot avatar mattops avatar michalnawrockidj avatar mikehalmamoj avatar mjwillis avatar neilmendum avatar petergphillips avatar simonmitchellmoj avatar sp-watson avatar steverendell avatar timharrison-moj avatar timoj58 avatar wbsouza avatar yasinmustafa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

prison-api's Issues

User access removed, access is now via a team

Hi there

The user emileswarts had Direct Member access to this repository and access via a team.

Access is now only via a team.

You may have less access it is dependant upon the teams access to the repo.

If you have any questions, please post in #ask-operations-engineering on Slack.

This issue can be closed.

Fix o.s.d.DataIntegrityViolationException in /api/offenders/

Expected Behavior

When the /api/offenders/ endpoint is called, an OFFENDERS record should be created in persistent storage.

Current Behavior

The system fails to create the OFFENDERS record...

...

Hibernate: insert into offenders (create_datetime, create_user_id, modify_datetime, modify_user_id, audit_additional_info, audit_client_ip_address, audit_client_user_id, audit_client_workstation_name, audit_module_name, audit_timestamp, audit_user_id, birth_date, caseload_type, create_date, race_code, first_name, sex_code, id_source_code, last_name, last_name_alpha_key, last_name_key, last_name_soundex, middle_name, middle_name_2, name_sequence, offender_id_display, root_offender_id, suffix, title, offender_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

...

org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:276)
...

Caused by: org.hsqldb.HsqlException: integrity constraint violation: NOT NULL check constraint; SYS_CT_10479 table: OFFENDERS column: CREATE_USER_ID
        at org.hsqldb.error.Error.error(Unknown Source)
        at org.hsqldb.Table.enforceRowConstraints(Unknown Source)
        at org.hsqldb.Table.generateAndCheckData(Unknown Source)
        at org.hsqldb.Table.insertSingleRow(Unknown Source)
        at org.hsqldb.StatementDML.insertSingleRow(Unknown Source)
        at org.hsqldb.StatementInsert.getResult(Unknown Source)
        at org.hsqldb.StatementDMQL.execute(Unknown Source)
        at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
        at org.hsqldb.Session.execute(Unknown Source)
        ... 175 common frames omitted

Context

The DDL for the CREATE_USER_ID column is...

...
CREATE_USER_ID                VARCHAR2(40) DEFAULT USER         NOT NULL
...

Steps to Reproduce

  1. Call the /api/offenders/ endpoint (see also OffenderResource#createPrisoner).

Explicitly define the prison-api project's Open Source Contribution policy

I'm on a non-ministryofjustice team. Our project will be consuming resources from the prison-api.

My job is to report to my team, answers to these questions:

  1. Is contributing to the prison-api project closed to non-ministryofjustice developers?
  2. If contributing to the prison-api project is not closed, what are the prerequisites for being allowed to contribute?
  3. Under what license are ministryofjustice projects in general (prison-api specifically) open sourced?
  4. In what sense of „open source" should ministryofjustice projects be thought of from a third-party project's perspective?

The Open Source Initiative mentions the word „license“ in nine of the ten numbered parts of its Open Source Definition:

  1. Free Redistribution
    The license shall not restrict any party...

  2. Source Code
    [not mentioned]

  3. Derived Works
    The license must allow modifications and derived works...

  4. Integrity of The Author's Source Code
    The license may restrict source-code from being distributed in modified form only if...

  5. No Discrimination Against Persons or Groups
    The license must not discriminate against any person or group of persons...

  6. No Discrimination Against Fields of Endeavor
    The license must not restrict anyone from making use of the program in a specific field...

  7. Distribution of License
    ...redistributed without the need for execution of an additional license...

  8. License Must Not Be Specific to a Product
    ...within the terms of the program's license...

  9. License Must Not Restrict Other Software
    The license must not place restrictions on other software...

  10. License Must Be Technology-Neutral
    No provision of the license may be predicated on any individual technology or style of interface...

However, the prison-api project doesn't have a license included or mentioned anywhere in it. My team and I are curious to understand what that means from a practical standpoint.

Where is hmpps-auth being set as the Authorization Server?

How does the prison-api specify hmpps-auth as its OAuth 2.0 authorization server? Where exactly in the code or configuration files I mean?

Or if you're doing it by Spring Security's auto-configuration, can you share a link to the Spring Security documentation that describes that auto-configuration? Please? I've Googled. But I couldn't find anything.

I'm aware of a few different ways provided by Spring Security for a resource server to configure what authorization server it will delegate OAuth 2.0 authorization to.

spring:
  security:
    oauth2:
      resourceserver:
        jwt:
          issuer-uri: https://idp.example.com/issuer
@EnableWebSecurity
public class DirectlyConfiguredJwkSetUri extends WebSecurityConfigurerAdapter {
    protected void configure(HttpSecurity http) {
        http
            .authorizeRequests(authorize -> authorize
                .anyRequest().authenticated()
            )
            .oauth2ResourceServer(oauth2 -> oauth2
                .jwt(jwt -> jwt
                    .jwkSetUri("https://idp.example.com/.well-known/jwks.json")
                )
            );
    }
}

I can't see anywhere in any prison-api file where the authorization server is set. Apart from this dev profile configurations I found in the project:

spring:
  security:
    oauth2:
      resourceserver:
        jwt:
          public-key-location: classpath:local-public-key.pub

And this test resource:

spring.security.oauth2.resourceserver.jwt.jwk-set-uri: http://localhost:9090/auth/.well-known/jwks.json

Thank you in advance.

A branch protection setting is not enabled: codeowners require reviews

Hi there
The default branch protection setting called codeowners require review is not enabled for this repository
This option affects a pull request, i.e a PR will need to be reviewed and approved by a CODEOWNER before it can be merged.
See repository settings/Branches/Branch protection rules
Either add a new Branch protection rule or edit the existing branch protection rule and select the Require review from Code Owners option
Create a .github/CODEOWNERS file
Add a or multiple entries of @ministryofjustice/team_name to the CODEOWNERS file
The team_name shall be a team from within the MoJ teams: https://github.com/orgs/ministryofjustice/teams
See GH Codeowners documentation: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
See the repository standards: https://github.com/ministryofjustice/github-repository-standards
See the report: https://operations-engineering-reports.cloud-platform.service.justice.gov.uk/github_repositories
Please contact Operations Engineering on Slack #ask-operations-engineering, if you need any assistance

Correct the principle username detection for OAuth 2.0 Client Credentials grant

Expected Behavior

When the /api/offenders/{offenderNo} endpoint is called, an OFFENDER_BOOKINGS record should be created in persistent storage.

Current Behavior

The system fails to create the OFFENDER_BOOKINGS record (full stack trace)...

...

org.springframework.dao.InvalidDataAccessApiUsageException: The given id must not be null!; nested exception is java.lang.IllegalArgumentException: The given id must not be null!

...

Caused by: java.lang.IllegalArgumentException: The given id must not be null!
        at org.springframework.util.Assert.notNull(Assert.java:201)
        at org.springframework.data.jpa.repository.support.SimpleJpaRepository.findById(SimpleJpaRepository.java:297)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:289)
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:137)
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:121)
        at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:529)
        at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285)
        at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:599)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java<===========--> 87% EXECUTING [5h 31m 35s]
        at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:13
...

Context

I am an external developer of a non-DPS system. My application is specified to consume API endpoints exposed by this project. My application will be authorized by HMPPS Auth using the OAuth 2.0 Client Credentials grant.

While stepping through the debugger during investigation of a separate, unrelated issue, I had observed that Spring Security refers to the security principle as "AnonymousUser" during the OAuth 2.0 Client Credentials grant flow.

Assuming that an "AnonymousUser" would not have a user name, then the following two lines are reasonable suspects for the root cause of the reported "The given id must not be null" error...

  1. (username = ((UserDetails) userPrincipal).getUsername();)
  2. staffUserAccountRepository.findById(currentUsername)

Steps to Reproduce

  1. Call the /api/offenders/{offenderNo} endpoint with a JWT access token issued by HMPPS Auth for an OAuth 2.0 Client Credentials grant.

Your Environment

  • Windows 10
  • Docker Desktop Version 3.3.3 (64133)
  • IntelliJ IDEA 2021.1.2 (Community Edition)
    • Build #IC-211.7442.40

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.