qinetiq-cyber-intelligence/opencti-terraform

A highly available AWS deployment of the Threat Intelligence platform, OpenCTI using Terraform. Native AWS resources are used where feasible.

OpenCTI AWS Deployment

A Terraform deployment of OpenCTI designed to make use of native AWS Resources (where feasible). This includes AWS ECS Fargate, AWS OpenSearch, AWS ElastiCache for Redis and AWS S3 (through a Gateway Endpoint).

Note This deployment is designed to help with OpenCTI Platform adoption. QinetiQ does not offer warranty on usage of this deployment. It is highly recommended to understand AWS, Terraform and Docker and if used within a production environment, perform an analysis of the deployment's security. This includes ensuring in production, that the state file is securely stored in S3 with a restrictive Bucket Policy. If storing credentials in the Terraform state file (in a locked down S3 Bucket) does not meet policy requirements, look into Terraform Environment variables to pass manually stored credentials or use AWS Secrets Manager that can be referenced in Terraform.

Requirements

This deployment requires

• Terraform AWS Provider Version >= 4.25.0
  • This is to make use of AWS EBS GP3 volumes, an important requirement to OpenCTI Platform performance.
• Terraform Version >= 1.1.0
• OpenCTI Platform Version >= 5.3.8
  • This deployment uses IAM Roles and AWS S3 Gateway Endpoint which requires the recent aws-sdk implementation that has been merged.

Key Features

• Regionally resilient with auto recovery capabilities

• Autoscaling OpenCTI Worker through AWS Lambda interacting with RabbitMQ metrics

• AWS SSM Jump Box solution to avoid SSH Keys

• Security conscious design

• OpenID Connect Implementation

• Scheduled Connectors capability (discussed in the OpenCTI Platform Connectors folder)

High-Level Architecture

Guidance

This Terraform deployment consists of two parts; deploying the core OpenCTI Platform and separately deploying the OpenCTI Connectors. This is to avoid the issue of redeploying the same Terraform deployment twice as OpenCTI Connectors should make use of their own OpenCTI User Account.

Design decisions for each deployment are covered within the respective folder's README.

OpenCTI URLs

• OpenCTI Website
• OpenCTI Demo
• OpenCTI GitHub
• OpenCTI Slack

Terraform Initialize

terraform init

Or in the case of using an S3 bucket to store Terraform State files.

1. Uncomment in versions.tf lines 8 - 10 to enable backend configuration and configure in ./config/dev/backend.conf the S3 bucket.

terraform init --backend-config=./config/dev/backend.conf

Terraform Deploy

terraform apply -var-file=config/dev/variables.tfvars

Terraform Tools

checkov

Note: When running checkov, it will fire a warning regarding an AWS WAF missing from the Application Load Balancer in the main OpenCTI Platform deployment. This is a resource you will need to add to this Terraform deployment.

Checkov is a tool used for checking static Terraform code against best security practices. To run locally, install checkov and then run checkov -d . --var-file=config/dev/variables.tfvars within either deployment.

tflint

tflint checks Terraform code against style guidelines. To run locally, install tflint, then run tflint --init and tflint.

License

This code is released under the Apache2 License. See LICENSE.

opencti-terraform's Issues

Failed on terraform apply steps

Steps to produce:

• clone repo: gh repo clone QinetiQ-Cyber-Intelligence/OpenCTI-Terraform
• terraform apply -var-file=config/dev/variables.tfvars

╷
│ Error: creating KMS Key: MalformedPolicyDocumentException: Policy contains a statement with one or more invalid principals.
│ 
│   with module.kms.aws_kms_key.this,
│   on modules/kms/main.tf line 87, in resource "aws_kms_key" "this":
│   87: resource "aws_kms_key" "this" {
│ 
╵
╷
│ Error: creating KMS Key: MalformedPolicyDocumentException: Policy contains a statement with one or more invalid principals.
│ 
│   with module.kms.aws_kms_key.connector,
│   on modules/kms/main.tf line 103, in resource "aws_kms_key" "connector":
│  103: resource "aws_kms_key" "connector" {
│ 
╵

Leaking credentials into terraform state

First of very nice deployment I am sure this will become a great starting point for people wanting to deploy their own instances.
I am aware the below describe issue is very nit picky, but this config is very well made and great contribution to the community so I want to help improve it even if its just a tiny thing.

When I went over your code I noticed that you are leaking credentials into terraform state in some places, most notably the opencti master password. Having the credentials in the terraform state introduces a few additional security risks e.g.,

• read permissions to the state file/bucket become will give access to all the secrets leaked into the state
• if used with a CI/CD pipeline anyone who can trigger a terraform plan is able to leak the secrets (see example below)

data "http" "example" {
  url = "http://localhost:8888/${var.test}"
}

variable "test" {
 default = "foobar"
 sensitive = true
}

$ terraform plan

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no
differences, so no changes are needed.

$ python3 -m http.server 8888
Serving HTTP on :: port 8888 (http://[::]:8888/) ...
::1 - - [19/Aug/2022 08:50:19] code 404, message File not found
::1 - - [19/Aug/2022 08:50:19] "GET /foobar HTTP/1.1" 404 -

Since this is not supposed to be a production ready deployment this is not an issue per say, but it might be worth mentioning this in the documentation so that people less familiar with terraform will be aware and modify the code accordingly when preparing for a production configuration.

Architecture diagram template?

Is there any chance to have the architecture diagram template uploaded into the assets as well?

Error creating OpenSearch domain: ValidationException: The StartAt time you provided occurs in the past.

Hello,

I am running into an issue as seen below. It looks like time_static only runs once, so subsequent deployment attempts cause an error.

Error: Error creating OpenSearch domain: ValidationException: The StartAt time you provided occurs in the past. Specify a time in the future.
│ 
│   with module.opensearch.aws_opensearch_domain.this,
│   on modules/opensearch/main.tf line 8, in resource "aws_opensearch_domain" "this":
│    8: resource "aws_opensearch_domain" "this" {

Load balancing - Question

This is really awesome work. The solution looks very slick. I can't use AWS however, I have to stick in Azure.

Can I ask how you are doing your load balancing? Specifically from the connectors to the OpenCTI frontend. I find with my current set up The front-end slows down and the RabbitMQ management portal show's that my consumer capacity is at 0% most of the time. I believe the GraphQL API is what's bottlenecking the system. I think replicating the OpenCTI front end will fix this but I am unsure how to load balance if I replicate the service.