Giter Site home page Giter Site logo

ranjanbinwani / dcos-universe Goto Github PK

View Code? Open in Web Editor NEW

This project forked from aquasecurity/dcos-universe

0.0 1.0 0.0 765 KB

Aqua Package For DC/OS Universe

Home Page: https://www.aquasec.com/

Makefile 1.14% Shell 15.62% HTML 17.61% Python 64.20% Dockerfile 1.43%

dcos-universe's Introduction

Aqua packages for DC/OS

This is an example configuration of an Aqua package repo for DC/OS, but this should not be used directly. Instead, clone this and customize it for your environment. Test thoroughly in non-production environment for your own use.

Contents


Quick Deployment Walk-through

This will walk through a complete deployment of Aqua console, gateway, and agents.

Step one: Duplicate this GitHub repository

Clone this GitHub repository for your own change control management. Replace our github repo location with your own. Alternatively, you can upload your own zip file to some alternate location.

Step two: Add Aqua repository to DC/OS

Add repository to DC/OS user interface by logging into DC/OS interface and browsing to System -> Repositories tab. Click the 'Add Repository button'.

Add repository

Include these details:

  • Name: AquaSecurity
  • URL: Zip file URL from repository in step one. Example, for this repo it would be https://github.com/aquasecurity/dcos-universe/archive/master.zip
  • Priority: 1

Click 'Add' to store it.

Browse to 'Universe' section from left hand menu. You should now have new packages:

Universe packages

Step three: Deploy database

Create a Postgres instance named 'aqua-db' by searching for 'Postgres' in the Universe.

Postgres

Change the service name to 'aqua-db':

aqua-db

You should set up persistent storage on the 'storage' section in left hand menu.

Click 'Review and Install' and then 'Install' to deploy the database.

You can confirm that the service is running on the Services tab.

Step four: Deploy Aqua console

When aqua-db is running, click back to Universe section and click 'Install' on 'aqua-web' and then 'Advanced Install'.

At a minimum, you will need to enter a license key.

aqua-web license

You will also need to decide how you will get the images into the environment. The Aqua images are hosted in private Docker Hub repositories, however you are free to push them to an internal registry if you like (this is a common enterprise scenerio).

DC/OS and Marathon has some interesting behavior around authentication to private registries. You can see this documented here.

Essentially, there are three options:

  • Push images to a registry that does not require authentication and then specify the image name in configuration settings.
  • Pre-pull the images on each server. Images will run from cache this way so there is no need to pull them again. Credentials can be removed after pull.
  • Create and distribute a docker config tarball per the Marathon documentation with a credential to Docker Hub that will allow access to the images.
  • Note: An example helper script named deployDockerCreds.sh is located in the ./scripts directory. Edit this script to match your environment.

The default option assumes use of pre-pulled images, but you can change the image name to include your registry or enable the docker config file and specify it's location on the 'docker' tab:

aqua-web docker configuration

This screen will be the same for other images as well.

Other settings like the default passwords and custom database hostnames can be set on the other tabs.

When configuration is set, click 'Review and Install' and then 'Install' to deploy aqua-web.

When you mouse over 'aqua-web' in the Services list, an external link icon will appear that will send you to the login page. Login here will be username and password. Validate that the aqua-web is running before continuing.

Step five: Deploy aqua-gateway

Click back through to Universe -> aqua-gateway -> Install.

If you are using the default options then you can just click 'Install' here.

Otherwise, if you have changed any settings such as the database service name, database passwords, or image name or deployment method, you can click 'Advanced Installation' to edit those settings. Then click 'Review and Install' and then 'Install' to deploy the gateways.

Go back to the Services tab. You should have running services now for everything except the agents:

Services

Step Six: Install the agents

Click back through to Universe -> aqua-agent -> Install, then 'Advanced Installation'.

Under the 'Instances' tab, set this to the number of nodes in the cluster to ensure agent is deployed everywhere. If this number (default: 3) is higher than the number of nodes, then there will always be tasks pending in the Services list for aqua-agent, but this will ensure the agent is automatically installed if you add a new node to the cluster.

You can also customize the same docker deployment options and other aqua config here. When set, click 'Review and Install' and then 'Install' to deploy the agents.

Step Seven: Verify installation

Click back through to Services -> aqua-web, and then click "Open Service" to get the login page for the aqua-web.

Login, and click the 'Hosts' section on the left hand side of the page. You should see the agents connected.

Hosts list

Daemon mode scanners (Scaling image scanning)

There is an additional package for the daemon-mode scanner-cli that can run standalone from aqua-web to provide greater throughput in image scanning.

Before you deploy, you should set up a dedicated scanning user in the Aqua user interface. To do this, browse to

System -> Users. Click the Create New User button at the top of the page.

On the resulting screen, enter a username, password (twice), and select the 'Scanner' role from the drop-down menu. Then click 'Save changes' to save the user.

Set up scanner user

The defaults used by the aqua-scanner service are username 'scanner' with password 'scanner123'.

To deploy, you can browse to Universe -> aqua-scanner -> Install. You can click 'Advanced Installation' to customize the username, password, or docker deployment settings (such as the image name).

The default number of scanner-cli instances is 3. This can be changed on the first 'service' screen in Advanced Installation:

aqua-scanner advanced install

Click Review and Install, and then Install to deploy.

You can verify that the scanners are deployed by going back to the Aqua console and browsing to Images -> Scan Queue (at top right, with arrow, may say "Scan Queue is empty" if there are no scans in progress).

The scanners will be listed on the right-hand side. By default there will be 1 scanner included in aqua web. If you added three in the aqua-scanner service then this will show 4 scanners total.

Scanner list

In DC/OS you can scale this up and down as needed on the Service page.

To do so, click Service -> aqua-scanner -> Scale button. You can set this to a higher or lower value to increase or decrease number of scanners.

Scale scanners

It can take several minutes for a scanner to disappear from the scan queue after it is removed, but new scanners will show up immediately.

Deployment Considerations

A few considerations should be taken into account.

Changes from default service names

If you change the service names from 'aqua-web', 'aqua-db', or 'aqua-gateway', you will need to do 'advanced install' options for all of the packages to change the addresses for the services that will be used, as it will use the DC/OS DNS service name instead of specific IPs for services.

Database persistence

Be sure to set persistent storage for the database component. External persistent storage is preferred, but this requires a plugin for Mesosphere.

Default passwords

The advanced install can also allow you to set non-default passwords (recommended, as defaults are just for demo and are insecure).

dcos-universe's People

Contributors

targaryen avatar jmichealson avatar ranlevko1 avatar oranmoshai avatar eranbibi avatar aquademouser avatar mehmetg avatar niso120b avatar jerbia avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.