Giter Site home page Giter Site logo

ministryofjustice / staff-device-dns-dhcp-admin Goto Github PK

View Code? Open in Web Editor NEW
1.0 12.0 3.0 6.58 MB

Web frontend for managing Staff Device DNS / DHCP servers

Home Page: https://github.com/ministryofjustice/cloud-operations#dhcp--dns

License: MIT License

Dockerfile 0.40% Ruby 79.45% Makefile 0.61% JavaScript 0.60% HTML 17.34% Shell 1.44% SCSS 0.16%

staff-device-dns-dhcp-admin's Introduction

repo standards badge .github/workflows/format-code.yml Brakeman Scan

Staff Device DNS / DHCP Admin

This is the web portal for managing Staff Device DNS / DHCP servers

Getting Started

Authenticate with AWS

Assuming you have been granted necessary access permissions to the Shared Service Account, please follow the NVVS DevOps best practices provided step-by-step guide to configure your AWS Vault and AWS Cli with AWS SSO.

Prepare the variables

  1. Clone the repository
  2. Copy .env.example to .env
  3. Modify the .env file and provide values for variables as below:
Variables How?
AWS_PROFILE= your AWS-CLI profile name for the Shared Services AWS account. Check this guide if you need help.
SHARED_SERVICES_ACCOUNT_ID= Account ID of the MoJO Shared Services AWS account.
REGISTRY_URL= <MoJO Development AWS Account ID>.dkr.ecr.eu-west-2.amazonaws.com
ENV= Your Terraform namespace from the DNS DHCP Infrastructure repo.
  1. Copy .env.development to .env.<your terraform namespace>

Prerequisite to starting the App

This repo is dependant on a locally running dhcp network. This is so that the admin app can query the dhcp api without timing out.

  1. Clone the repository here
  2. Follow the instructions in the cloned repository to run the dhcp server
  3. Navigate back to this repo

Starting the App

  1. If this is the first time you have setup the project:

    1. Build the base containers

      make build-dev
    2. Setup the database

      make db-setup
  2. Start the application

$ make serve

Running Tests

  1. Setup the test database
make db-setup
  1. Run the entire test suite
make test

To run individual tests:

  1. Shell onto a test container
ENV=test make shell
  1. Run the test file or folder
bundle exec rspec path/to/spec/file

Scripts

There are two utility scripts in the ./scripts directory to:

  1. Migrate the database schema
  2. Deploy new tasks into the service

Deployment

The deploy command is wrapped in a Makefile. It calls ./scripts/deploy which schedules a zero downtime phased deployment in ECS.

It doubles the currently running tasks and briefly serves traffic from the new and existing tasks in the service. The older tasks are eventually decommissioned, and production traffic is gradually shifted over to only the new running tasks.

On CI this command is executed from the buildspec.yml file after migrations and publishing the new image to ECR has been completed.

Targetting the ECS Cluster and Service to Deploy

The ECS infrastructure is managed by Terraform. The name of the cluster and service are outputs from the Terraform apply. These values are published to SSM Parameter Store, when this container is deployed it pulls those values from Parameter Store and sets them as environment variables.

The deploy script references these environment variables to target the ECS Admin service and cluster. This is to avoid depending on the hardcoded strings.

The build pipeline assumes a role to access the target AWS account.

Publishing Image from Local Machine

  1. Export the following configurations as an environment variable.
  export DHCP_DNS_TERRAFORM_OUTPUTS='{
    "admin": {
      "ecs": {
        "cluster_name": "[TARGET_CLUSTER_NAME]",
        "service_name": "[TARGET_SERVICE_NAME]"
      }
    }
  }'

This mimics what happens on CI where this environment variable is already set.

When run locally, you need to target the AWS account directly with AWS Vault.

  1. Schedule the deployment
  aws-vault exec [target_aws_account_profile] -- make deploy

Maintenance

AWS RDS SSL Certificate

The AWS RDS SSL certificate is due to expire August 22, 2024. See the documentation for information on updating the certificate closer to the date.

To update the certificate, update the Dockerfile to use the new intermediate (region specific) certificate (found here), and update the config/database.yml to point to the new certificate file path.

DHCP Data Checks

For information on how to perform the data import before network cutover, please see the documentation.

CI/CD

Known Issues

  • Dependabot currently does not support a container image monitoring solution only for the Docker container ruby:3.2.2-alpine3.16, this alpine images needs to be updated manually.

staff-device-dns-dhcp-admin's People

Contributors

astrobinson avatar bagg3rs avatar caitbarnard avatar darey-io avatar dependabot[bot] avatar efuaakum avatar efuaakumanyi avatar emileswarts avatar gary-h9 avatar github-actions[bot] avatar jamesgreen-moj avatar jbevan4 avatar jivdhaliwal avatar juddin927 avatar laurentb4 avatar mitchdawson1982 avatar mtouhid avatar neilkidd avatar paulmchenry avatar richrace avatar satishgummadellimoj avatar smjmoj avatar themitchell avatar wanieldilson avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

staff-device-dns-dhcp-admin's Issues

Improve API Performance for DHCP Stats

Steps:

💂‍♀️ SPIKE: Sentry TLS version

"Hi Ministry of Justice,
As you may have seen, Sentry is planning to retire support for TLS 1.0 and 1.1.
Our records show that your organization has some projects using TLS 1.0.
We encourage you to upgrade some of your systems to support TLS 1.2+. We know that in some situations, it may not be possible to switch to newer TLS versions, so alternatively, we ask that you update your Sentry SDK configuration to allow TLS 1.0 usage.
If you are using the domain oXXX.ingest.sentry.io in your Sentry SDK configuration, then you can update it to oXXX.insecure.sentry.io to continue using TLS 1.0.
If you have any questions, please feel free to reply directly to this email.
The Sentry Team"

⚠️ `already initialized constant Net::` error with rails 7.0.1 on ruby 2.7.1

After the rails been upgraded to version 7.0.1, we are getting the below warnings when running RSpec tests locally:

/usr/local/lib/ruby/2.7.0/net/protocol.rb:66: warning: already initialized constant Net::ProtocRetryError
/usr/local/bundle/gems/net-protocol-0.1.2/lib/net/protocol.rb:68: warning: previous definition of ProtocRetryError was here
/usr/local/lib/ruby/2.7.0/net/protocol.rb:206: warning: already initialized constant Net::BufferedIO::BUFSIZE
/usr/local/bundle/gems/net-protocol-0.1.2/lib/net/protocol.rb:208: warning: previous definition of BUFSIZE was here
/usr/local/lib/ruby/2.7.0/net/protocol.rb:503: warning: already initialized constant Net::NetPrivate::Socket
/usr/local/bundle/gems/net-protocol-0.1.2/lib/net/protocol.rb:504: warning: previous definition of Socket was here

In order to avoid those errors, we need to upgrade ruby to 3.0.2.

Create New Non Res Wifi Subnets

Created new subnets as per the request from the Non Res Wifi rollout team.

Site Site allocation Staff Wireless Subnet FITS ID Domain Name Router IP Subnet Mask Exclusion Range Notes
Cookham Wood 10.154.32.0/21 10.154.32.0/23 FITS_4175 a03-COO-1.a03.wp360g.svcs.hp.com 10.154.32.1 255.255.254.0 10.154.32.1 - 5 Exclusion amended 26/1/23
Erlestoke 10.154.48.0/21 10.154.48.0/23 FITS_0990 " a03-ERL-1.a03.wp360g.svcs.hp.com" 10.154.48.1 255.255.254.0 10.154.48.1 - 5 Exclusion amended 26/1/23
Feltham 10.154.64.0/21 10.154.64.0/23 FITS_0879 " a03-YFE-1.a03.wp360g.svcs.hp.com" 10.154.64.1 255.255.254.0 10.154.64.1 - 5 Exclusion amended 26/1/23
Garth 10.154.80.0/21 10.154.80.0/23 FITS_1032 a03-GAR-1.a03.wp360g.svcs.hp.com 10.154.80.1 255.255.254.0 10.154.80.1 - 5 Exclusion amended 26/1/23
Lindholme 10.154.96.0/21 10.154.96.0/23 FITS_0897 a03-LIH-1.a03.wp360g.svcs.hp.com 10.154.96.1 255.255.254.0 10.154.96.1 - 5 Exclusion amended 26/1/23
Newhall 10.154.112.0/21 10.154.112.0/23 FITS_0877 " a03-NEW-1.a03.wp360g.svcs.hp.com" 10.154.112.1 255.255.254.0 10.154.112.1 - 5 Exclusion amended 26/1/23
Ranby 10.154.128.0/21 10.154.128.0/23 FITS_0921 a03-RAN-1.a03.wp360g.svcs.hp.com 10.154.128.1 255.255.254.0 10.154.128.1 - 5 Exclusion amended 26/1/23
Stoke Heath 10.154.144.0/21 10.154.144.0/23 FITS_1033 " a03-YST-1.a03.wp360g.svcs.hp.com" 10.154.144.1 255.255.254.0 10.154.144.1 - 5 Exclusion amended 26/1/23
Styal 10.154.16.0/21 10.154.16.0/23 FITS_0862 " a03-YSY-1.a03.wp360g.svcs.hp.com" 10.154.16.1/23 255.255.254.0 10.154.16.1 - 5 Exclusion amended 26/1/23
Swaleside 10.154.160.0/21 10.154.160.0/23 FITS_2201 a03-SWA-1.a03.wp360g.svcs.hp.com 10.154.160.1/23 255.255.254.0 10.154.160.1 - 5 Exclusion amended 26/1/23
The Mount 10.154.176.0/21 10.154.176.0/23 FITS_1039 " a03-THM-1.a03.wp360g.svcs.hp.com" 10.154.176.1/23 255.255.254.0 10.154.176.1 - 5 Exclusion amended 26/1/23
Werrington 10.154.0.0/21 10.154.0.0/23 FITS_0959 None currently configured 10.154.0.1/23 255.255.254.0 10.154.0.1 - 5 Exclusion amended 26/1/23
Wetherby 10.154.192.0/21 10.154.192.0/23 " FITS_0890" " a03-WET-1.a03.wp360g.svcs.hp.com" 10.154.192.0/23 255.255.254.0 10.154.192.1 - 5 Exclusion amended 26/1/23

Test Issues - Does the new GtiHub Org secret work...

User Story

As a…
I need/want/expect to…
So that…

Value / Purpose

No response

Useful Contacts

No response

Additional Information

No response

Definition of Done

Example

  • Documentation has been written / updated
  • README has been updated
  • User docs have been updated
  • Another team member has reviewed
  • Tests are green

📊 User Metrics

User Story

As a… Cloud Ops Engineer
I need/want/expect to… understand how many users have logged into the portal in the last day/week/month/year
So that… we can better understand the use of the portal / liaise with the rest of the business with this information

Value / Purpose

As per the recent road shows the business expects teams / products to be data driven, this metric will tie in with that.

Useful Contacts

NA

Additional Information

NA

Definition of Done (DoD)

As part of refinement it should be decided if the metrics should appear on the port under a new route e.g. /metrics or /stats OR the data should appear in a Grafana dashboard. It should also be decided if this should be viewable by all users or just the admin if added to a route.

Checklist for definition of done and acceptance criteria, example below (optional).

Test Story Issue

User Story

As a…
I need/want/expect to…
So that…

Value / Purpose

No response

Useful Contacts

No response

Additional Information

No response

Definition of Done

Example

  • Documentation has been written / updated
  • README has been updated
  • User docs have been updated
  • Another team member has reviewed
  • Tests are green

🐛 Makefile errors

Describe the bug.

Makefile needs to be updated post our move from AWS ECR to Docker Hub.

To Reproduce

  1. Pull main.
  2. run make authenticate-docker.
  3. Receive unbound variable error.

Expected Behaviour

No response

Environment

No response

Additional context

No response

Collaborator review date expires soon for user emileswarts

Hi there

The user @emileswarts has its access for this repository maintained in code here: https://github.com/ministryofjustice/github-collaborators

The review_after date is due to expire within one month, please update this via a PR if they still require access.

If you have any questions, please post in #ask-operations-engineering on Slack.

Failure to update the review_date will result in the collaborator being removed from the repository via our automation.

User access removed, access is now via a team

Hi there

The user emileswarts had Direct Member access to this repository and access via a team.

Access is now only via a team.

You may have less access it is dependant upon the teams access to the repo.

If you have any questions, please post in #ask-operations-engineering on Slack.

This issue can be closed.

✨ DHCP Portal search feature

User Story

As a user of the DHCP admin interface
I want to easily locate a subnet's site
So that I don't waste time manually looking at every site to find a given subnet

Value / Purpose

No response

Useful Contacts

No response

Additional Information

No response

Definition of Done

Example

  • Documentation has been written / updated
  • README has been updated
  • User docs have been updated
  • Another team member has reviewed
  • Tests are green

⏰ Alert for subnet capacity

User Story

As a CloudOps Engineer
I want an alert when a DHCP pool reaches 90% capacity
So that we can be more proactive with issues on MoJo sites

Value / Purpose

To increase productivity regarding DHCP issues on MoJO sites

Useful Contacts

[Please add any useful contacts, these may include: Stakeholders, SME’s or 3rd Parties]

Additional Information

[Please add any useful links or additional information that would be beneficial to anyone working on this ticket]

Definition of Done (DoD)

AlertManager sends alerts to a Slack channel when any subnet reaches 90% capacity

:sparkles: Add custom DHCP option 234 in Kea Config

User Story

As a… Developer
I want Admin Portal app to… add custom DHCP option 234 def and data in the config file for Kea
So that… Whenever a new config gets generated, the new custom DHCP option is always present for all subnets

Value / Purpose

This new feature enables the MoJ OFFICIAL EUC devices to better share application and Windows Update payloads across the local site network without going out to the WAN and Internet.

Useful Contacts

Matt White, Chandra Singh, Charlie Coverdale

Additional Information

No response

Definition of Done

Example

  • Documentation has been written / updated
  • README has been updated
  • User docs have been updated
  • Another team member has reviewed
  • Tests are green

⏰ No Data Alerting - Rate Metrics

User Story

As a… cloud ops engineer
I need/want/expect to… be alerted when the rate metrics in this dashboard have no data.
So that… I am aware of issues relating to the rate metrics not being picked up.

Value / Purpose

Post an incident we noted that these metrics were no data and we had not been alerted. This is not acceptable and this ticket should resolve this.

Definition of Done (DoD)

Alerts are triggered when the following metrics provide no data.
image

Collaborator review date expires soon for user emileswarts

Hi there

The user @emileswarts has its access for this repository maintained in code here: https://github.com/ministryofjustice/github-collaborators

The review_after date is due to expire within one month, please update this via a PR if they still require access.

If you have any questions, please post in #ask-operations-engineering on Slack.

Failure to update the review_date will result in the collaborator being removed from the repository via our automation.

User access removed, access is now via a team

Hi there

The user emileswarts had Direct Member access to this repository and access via a team.

Access is now only via a team.

You may have less access it is dependant upon the teams access to the repo.

If you have any questions, please post in #ask-operations-engineering on Slack.

This issue can be closed.

[URGENT] <Add a new zone to MOJO DNS for azure.hmpp.root >

User Story

Due to DXC exit infrastructure had been decommissioned and resulted azure.hmpp.root unresolvable. As a part of the major incident response we have been asked to add below record urgently in order to direct DNS queries to Azure Landing Zone

Value / Purpose

azure.hmpp.root 10.40.0.13310.40.128.196 forward queries to Azure Landing Zones

Useful Contacts

Matt White

Additional Information

Major incident ref INC2105932

Definition of Done

  • Zone created and infrastructure redeployed
  • Another team member has reviewed
  • Tests are green

Collaborator review date expires soon for user emileswarts

Hi there

The user @emileswarts has its access for this repository maintained in code here: https://github.com/ministryofjustice/github-collaborators

The review_after date is due to expire within one month, please update this via a PR if they still require access.

If you have any questions, please post in #ask-operations-engineering on Slack.

Failure to update the review_date will result in the collaborator being removed from the repository via our automation.

DHCP Option 234 - Inform responses being dropped.

**** This ticket serves for information purposes only. ****

The devops team were approached by the EUC team with regards to issues with clients obtaining dhcp option 234. After some testing it became apparent that windows clients first obtained an ip address via a normal discover, offer, request, ack (DORA) flow, however sometime afterwards the client would request parameter 234 via a dhcp inform message. The difference with inform messages is that the response from the dhcp server is sent back directly to the client rather than via the dhcp relay ip. It was suspected that responses from the server were being dropped somewhere as the client never received it. After testing and validating the policies and logs in the 10sc model office environment it was found that there were no firewall rules permitting responses from the dhcp server subnets back to client subnets and the responses were therefore being denied.

🐛 `mysql client library does not support ssl_mode` noted during pipeline run

Describe the bug.

/usr/local/bundle/gems/mysql2-0.5.3/lib/mysql2/client.rb:51: warning: Your mysql client library does not support ssl_mode as expected.

To Reproduce

Look at the DHCP Admin pipeline. This error occurs at the start of every run in each environment.

Expected Behaviour

Error should not occur, needs to be investigated.

Environment

No response

Additional context

No response

:wrench: - Replace hardcoded variables with terraform outputs

User Story

As a CloudOps engineer I expect our code to pull variables from the terraform outputs rather than being hardcoded in scripts so that any changes to names and or id's are automatically handled.

Value / Purpose

By removing hardcoded dependencies in our code we can simplify and future proof our systems going forward.

Useful Contacts

Mitch Dawson

Additional Information

/scripts/bootstrap.sh
/scripts/migrate.sh
/scripts/seed.sh

Example:

cluster_name="staff-device-${ENV}-dhcp-admin-cluster"
service_name="staff-device-${ENV}-dhcp-admin"
task_definition="staff-device-${ENV}-dhcp-admin-task"

Definition of Done

Example

  • Affected scripts have been reviewed and updated
  • Documentation has been written / updated
  • Another team member has reviewed
  • Tests are green

⚠️ Set Rate Limit on Sentry.io

User Story

As a Sentry.io user
I want to limit the rate of errors reported to sentry.io
So that we as a team do not use up the transaction quota for MoJ

Value / Purpose

Sentry.io is a shared service across many projects and due to one team not having configured rate limiting a single project has used up the monthly error transaction quota (about 90% in a single day). - let's not be that team.

Useful Contacts

Aaron Robinson

Additional Information

original slack message: https://mojdt.slack.com/archives/C02D2NEF9CJ/p1657787053509449

Sentry.io rate limiting

Cloud Platform guidance.

Definition of Done

Rate limiting correctly configured on Sentry.io for our projects:

  • staff-device-dns-dhcp-admin
  • staff-device-dhcp-metrics-agent
  • staff-device-dns-metrics-agent

⬆️ DNS DHCP Admin Portal MySQL Upgrade Issue.

User Story

As a… Cloud Ops Engineer
I want to… ensure MySQL dependency upgrades are successfully tested locally before being pushed through the pipeline.
So that… i can be confident that no downtime will arise from such upgrades.

Value / Purpose

The purpose of this ticket is to review testing & documentation for Dependabot (MySQL) upgrades. The following PR (499) passed all local development testing but failed to deploy successfully to Dev / Pre-Prod through the pipeline.

There are any...

  • Additional tests that could be written
  • Database upgrade documentation

Useful Contacts

Aaron Robinson (Cloud Ops Team)

Additional Information

#499

https://sentry.io/organizations/ministryofjustice/issues/3247582520/?project=5458370&referrer=slack

Definition of Done

  • Documentation has been written / updated
  • Another team member has reviewed
  • Tests are green

Demo Ticket

User Story

As a…
I need/want/expect to…
So that…

Value / Purpose

No response

Useful Contacts

No response

Additional Information

No response

Definition of Done

Example

  • Documentation has been written / updated
  • README has been updated
  • User docs have been updated
  • Another team member has reviewed
  • Tests are green

⬆️ Migrate to 'sentry-ruby' from 'sentry-raven' as it is now deprecated

User Story

As a Cloud Ops engineer
I expect to be using the latest SDKs (software development kit)
So that our platforms/applications arent' missing important fixes and features.

Value / Purpose

Purpose:
You have 1 project using a deprecated version of the Sentry client. Any SDK that has the package name ‘raven’ may be missing data. Migrate to the latest SDK version.

Value:
Benefits
Unified Interfaces With Other SDKs: The design of sentry-raven is outdated compared with our modern Sentry SDKs. If you also use other Sentry SDKs, such as Sentry's JavaScript SDK for your frontend application, you'll notice that their interfaces are quite different from the one provided for sentry-raven. The new sentry-ruby SDK provides a more consistent user experience across all different platforms.

Performance Monitoring: The Sentry Ruby SDK includes performance monitoring, which you can enable if you haven't already as (discussed here).

Future Support: sentry-raven has entered maintenance mode, which means it won't receive any new feature supports or aggressive bug fixes.

Better Extensibility: Unlike sentry-raven, sentry-ruby is built with extensibility in mind and will allow the community to build extensions for different integrations/features

Useful Contacts

Aaron Robinson

Additional Information

Migration Guide: https://docs.sentry.io/platforms/ruby/migration/

Definition of Done

staff-device-dns-dhcp-admin is migrated from 'sentry-raven' to 'sentry-ruby' and the reminder for the update is no longer present on sentry.io

  • Tests are green

📊 SPIKE: Data Driven Metrics

User Story
As a… Cloud Ops Engineer
I need/want/expect to… provide metrics relating to the portal to the business
So that… we can better understand the use of the portal / liaise with the rest of the business with this information

Value / Purpose
As per the recent road shows the business expects teams / products to be data driven, this metric will tie in with that. This spike will allow us to think of all the metrics we would like to have in future stories.

Definition of Done (DoD)
Provide a list of metrics which later stories can be made from.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.