Giter Site home page Giter Site logo

bishopfox / dufflebag Goto Github PK

View Code? Open in Web Editor NEW
269.0 12.0 40.0 853 KB

Search exposed EBS volumes for secrets

License: GNU General Public License v3.0

Makefile 0.01% Go 100.00%
aws-ebs aws-ebs-volumes aws-ebs-snapshot aws-eb elasticbeanstalk security-tools

dufflebag's Introduction

Dufflebag

Dufflebag is a tool that searches through public Elastic Block Storage (EBS) snapshots for secrets that may have been accidentally left in. You may be surprised by all the passwords and secrets just laying around!

The tool is organized as an Elastic Beanstalk ("EB", not to be confused with EBS) application, and definitely won't work if you try to run it on your own machine.

Dufflebag has a lot of moving pieces because it's fairly nontrivial to actually read EBS volumes in practice. You have to be in an AWS environment, clone the snapshot, make a volume from the snapshot, attach the volume, mount the volume, etc... This is why it's made as an Elastic Beanstalk app, so it can automagically scale up or down however much you like, and so that the whole thing can be easily torn down when you're done with it.

Just keep an eye on your AWS console to make sure something it's going haywire and racking up bills. We've tried to think of every contingency and provide error handling... but you've been warned!

Getting Started

Permissions

You'll need to add some additional AWS IAM permissions to the role: aws-elasticbeanstalk-ec2-role. Alternatively, you can make a whole new role with these permissions and set EB to use that role, but it's a little more involved. In any case, you'll need to add:

  • AttachVolume (ec2)
  • CopySnapshot (ec2)
  • CreateVolume (ec2)
  • DeleteSnapshot (ec2)
  • DeleteVolume (ec2)
  • DescribeSnapshots (ec2)
  • DescribeVolumes (ec2)
  • DetachVolume (ec2)
  • PurgeQueue (sqs)
  • ListQueues (sqs)
  • ListAllMyBuckets (s3)
  • PutObject (s3)

Building

The core application is written in Go, so you'll need a Golang compiler. But the EB application is actually built into a .zip file (that's just how EB works) so the makefile will output a zip for you.

  1. Check your region. Dufflebag can only operate in one AWS region at a time. If you want to search every region, you'll have to deploy that many instances. To change the region, change the contents of the source code file region.go.

  2. Install dependencies: Ubuntu 18.04 x64:

sudo apt install make golang-go git
go get -u github.com/aws/aws-sdk-go
go get -u github.com/deckarep/golang-set
go get -u github.com/lib/pq
go get -u lukechampine.com/blake3
  1. Then build the EB app into a zip file with:
make

You should now see a dufflebag.zip file in the root project directory.

  1. Lastly, you'll need to make an S3 bucket. Setting this up automatically within Dufflebag might be possible, but it'd actually be pretty hard. So just do it yourself. You just need to make an S3 bucket with default permissions, and have the name start with dufflebag. S3 bucket names have to be globally unique, so you'll probably need to have some suffix that is a bunch of gibberish or something.

Deploying to Elastic Beanstalk

Go to your AWS console and find the Elastic Beanstalk menu option. This is probably doable via the CLI too, but this description will use the console. Select Actions -> Create Environment.

Create Environment

Then in the next window choose Worker environment and hit Select.

Worker Environment

In the next window, choose Preconfigured -> Go for the Platform.

Go  Platform

Under Application Code, choose Upload your Code.

Upload You Code

Hit the Upload button and select the dufflebag.zip that you just built.

Code Origins

Finally, hit Create Environment to get starting.

It will take a few minutes for AWS to make all the things and get started. Once created, Dufflebag will get started right away. No need to do anything else.

Remove the Safety Valve

Once you have this up and running, you can try again with the safety valve removed. By default, Dufflebag only searches 20 EBS snapshots. (So that it doesn't go haywire on your very first try) In order to widen the search to the entire region, go into populate.go, remove the following line of code, and rebuild:

//#####################################################################
//####                    Safety Valve                             ####
//#### Remove this line of code below to search all of your region ####
//#####################################################################
snapshots = snapshots_result.Snapshots[0:20]

Scaling Up

One of the reasons Dufflebag is designed as an Elastic Beanstalk app is so that you can automatically scale the program up or down easily. By default, it'll just run on one instance and be pretty slow. But if you'd like to juice it up a little, adjust the autoscaling in Elastic Beanstalk. The full options of this are a little outside the scope of this document, so I'll let you play with them. But in practice, I've found that a simple CPU use trigger works pretty well.

When setting up the environment above, you'll find the options under the Configure more options button (instead of hitting Create environment) and then hit Scaling.

Scaling

Getting The Stolen Goods

Dufflebag will copy out any interesting files out to the S3 bucket that you made earlier. (Technically, Dufflebag will use the first S3 bucket it finds who's name starts with "dufflebag".)

You can just watch the files come in one-by-one in your S3 bucket. They will be named:

originalfilename_blake3sum_volumeid

Checking on Status

If everything is going well, you shouldn't need to read the logs. But just in case, Elastic Beanstalk lets apps write to log files as they run, and these are captured by navigating to the Logs tab. Then hit Request Logs and Last 100 Lines. This will get you the most recent batch of Dufflebag logs. Hit the Download button to read it. This file will contain a bunch of other system logs, but the Dufflebag part is under "/var/log/web-1.log" at the top.

In order to see the full log history, select Full Logs instead of Last 100 Lines. (Note that EB will rotate logs away pretty aggressively by default)

Additionally, you can get a sense of overall progress by looking at the SQS queue for the environment. Elastic Beanstalk worker environments use SQS to manage the workflow. Each message in the queue for Dufflebag represents an EBS volume to process:

Code Origins

The Messages Available column shows how many volumes have not yet been processed. The Messages in Flight column shows how many volumes are being processed right now.

Tweaking what to Search For

Dufflebag is programmed to search for stuff we thought would be likely to be "interesting". Private keys, passwords, AWS keys, etc... But what if you really want to search for something specific to YOU? Like, maybe you work for bank.com and would like to see what's out there that references bank.com.

Doing so will require a minor amount of modification to the Dufflebag code, but not very much. Don't worry! The logic for what to search for happens in inspector.go. The pilfer() function is a goroutine that handles inspecting a file. The code there may look a little intimidating at first, but here's what it's doing. (And how you can modify that without much difficulty)

File name blacklists:

  1. Check the file name against a blacklist. (blacklist_exact)
  2. Check the file name against a "contains" blacklist. (Reject the file if it contains a given string) (blacklist_contains)
  3. Check the file name against a prefix blacklist. (Reject the file if it starts with a given string) (blacklist_prefix)

You can modify the searching logic here pretty easily by just changing what's in those three lists. Though I'd in general recommend leaving these in tact. These blacklists are designed to cover boring files that are present in a lot of filesystems and prevents Dufflebag from needing to inspect in-depth every single file on all of AWS. Sensitive data you're looking for MIGHT be in those files... but probably not.

File name whitelist:

  1. The IsSensitiveFileName() function checks the file name against a regular expression that finds sensitive file names. (Such as /etc/shadow, bash_history, etc...)

File contents:

  1. The function checkContentsRegex() checks the file contents against a set of regular expressions. (The file input argument is line-by-line, so the input to this function is one line of a file, not the whole file.) So to look for keywords related to your organization, just change up the regular expressions inside checkContentsRegex().

dufflebag's People

Contributors

dan-bishopfox avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dufflebag's Issues

Error while launching the application

  • % of customers impacted: [100]%
  • Affected Users: [Engineers / Operators / Customers]
  • Is revenue impacted: [Yes / No]
  • Urgency: [High / Medium / Low]
  • Deadline: [Date]

Description of Bug

Instance is not launching every single time we configure the application

What should the expected behavior be

Platform Affected

[ ] Dev
[ ] Preview
[ ] Production

Steps to Reproduce

Steps to reproduce the behavior

Additional context

From AWS account.

Relevant screenshots

Error logs from EBS events:

Create environment operation is complete, but with errors. For more information, see troubleshooting documentation.

[Instance: i-09b310f4235bfc2e9] Command failed on instance. Return code: 1 Output: Engine execution has encountered an error..

Instance deployment failed. For details, see 'eb-engine.log'.

Relevant eb-engine.log


/var/log/web.stdout.log

Sep 7 07:55:56 ip-172-31-44-139 web: /bin/sh: application: command not found
Sep 7 07:55:57 ip-172-31-44-139 web: /bin/sh: application: command not found
Sep 7 07:55:57 ip-172-31-44-139 web: /bin/sh: application: command not found
Sep 7 07:55:57 ip-172-31-44-139 web: /bin/sh: application: command not found
Sep 7 07:55:58 ip-172-31-44-139 web: /bin/sh: application: command not found


/var/log/eb-engine.log

2023/09/07 07:55:54.901081 [INFO] Executing instruction: configure healthd specific proxy conf
2023/09/07 07:55:54.902815 [INFO] Running command /bin/sh -c systemctl show -p PartOf healthd.service
2023/09/07 07:55:54.909056 [INFO] Running command /bin/sh -c systemctl daemon-reload
2023/09/07 07:55:54.974719 [INFO] Running command /bin/sh -c systemctl reset-failed
2023/09/07 07:55:54.979509 [INFO] Running command /bin/sh -c systemctl is-enabled aws-eb.target
2023/09/07 07:55:54.983062 [INFO] Running command /bin/sh -c systemctl enable aws-eb.target
2023/09/07 07:55:55.055425 [INFO] Running command /bin/sh -c systemctl start aws-eb.target
2023/09/07 07:55:55.060697 [INFO] Running command /bin/sh -c systemctl enable healthd.service
2023/09/07 07:55:55.136012 [INFO] Created symlink from /etc/systemd/system/multi-user.target.wants/healthd.service to /etc/systemd/system/healthd.service.

2023/09/07 07:55:55.136078 [INFO] Running command /bin/sh -c systemctl show -p PartOf healthd.service
2023/09/07 07:55:55.141505 [INFO] Running command /bin/sh -c systemctl is-active healthd.service
2023/09/07 07:55:55.144972 [INFO] Running command /bin/sh -c systemctl start healthd.service
2023/09/07 07:55:55.167960 [INFO] Copying file /opt/elasticbeanstalk/config/private/healthd/healthd_logformat.conf to /var/proxy/staging/nginx/conf.d/healthd_logformat.conf
2023/09/07 07:55:55.168945 [INFO] Copying file /opt/elasticbeanstalk/config/private/healthd/healthd_nginx.conf to /var/proxy/staging/nginx/conf.d/elasticbeanstalk/healthd.conf
2023/09/07 07:55:55.170041 [INFO] Executing instruction: configure log streaming
2023/09/07 07:55:55.170050 [INFO] log streaming is not enabled
2023/09/07 07:55:55.170054 [INFO] disable log stream
2023/09/07 07:55:55.170068 [INFO] Running command /bin/sh -c systemctl show -p PartOf amazon-cloudwatch-agent.service
2023/09/07 07:55:55.179984 [INFO] Running command /bin/sh -c systemctl stop amazon-cloudwatch-agent.service
2023/09/07 07:55:55.189588 [INFO] Executing instruction: GetToggleForceRotate
2023/09/07 07:55:55.189612 [INFO] Checking if logs need forced rotation
2023/09/07 07:55:55.189631 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:us-west-2:147985732667:stack/awseb-e-wpy2emnhff-stack/d095b300-4d53-11ee-a7a3-0afc390dc713 -r AWSEBAutoScalingGroup --region us-west-2
2023/09/07 07:55:55.744439 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:us-west-2:147985732667:stack/awseb-e-wpy2emnhff-stack/d095b300-4d53-11ee-a7a3-0afc390dc713 -r AWSEBBeanstalkMetadata --region us-west-2
2023/09/07 07:55:56.214490 [INFO] Generating rsyslog config from Procfile
2023/09/07 07:55:56.215976 [INFO] Running command /bin/sh -c systemctl restart rsyslog.service
2023/09/07 07:55:56.249176 [INFO] Executing instruction: PostBuildEbExtension
2023/09/07 07:55:56.249205 [INFO] Starting executing the config set Infra-EmbeddedPostBuild.
2023/09/07 07:55:56.249227 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-init -s arn:aws:cloudformation:us-west-2:147985732667:stack/awseb-e-wpy2emnhff-stack/d095b300-4d53-11ee-a7a3-0afc390dc713 -r AWSEBAutoScalingGroup --region us-west-2 --configsets Infra-EmbeddedPostBuild
2023/09/07 07:55:56.610416 [INFO] Finished executing the config set Infra-EmbeddedPostBuild.

2023/09/07 07:55:56.610497 [INFO] Executing instruction: CleanEbExtensions
2023/09/07 07:55:56.610713 [INFO] Cleaned ebextensions subdirectories from app staging directory.
2023/09/07 07:55:56.610720 [INFO] Executing instruction: RunAppDeployPreDeployHooks
2023/09/07 07:55:56.610735 [INFO] Executing platform hooks in .platform/hooks/predeploy/
2023/09/07 07:55:56.610751 [INFO] The dir .platform/hooks/predeploy/ does not exist
2023/09/07 07:55:56.610753 [INFO] Finished running scripts in /var/app/staging/.platform/hooks/predeploy
2023/09/07 07:55:56.610759 [INFO] Executing instruction: stop X-Ray
2023/09/07 07:55:56.610762 [INFO] stop X-Ray ...
2023/09/07 07:55:56.610771 [INFO] Running command /bin/sh -c systemctl show -p PartOf xray.service
2023/09/07 07:55:56.619817 [WARN] stopProcess Warning: process xray is not registered
2023/09/07 07:55:56.619841 [INFO] Running command /bin/sh -c systemctl stop xray.service
2023/09/07 07:55:56.626763 [INFO] Executing instruction: stop proxy
2023/09/07 07:55:56.626792 [INFO] Running command /bin/sh -c systemctl show -p PartOf httpd.service
2023/09/07 07:55:56.630915 [WARN] deregisterProcess Warning: process httpd is not registered, skipping...

2023/09/07 07:55:56.630956 [INFO] Running command /bin/sh -c systemctl show -p PartOf nginx.service
2023/09/07 07:55:56.636551 [WARN] deregisterProcess Warning: process nginx is not registered, skipping...

2023/09/07 07:55:56.636569 [INFO] Executing instruction: FlipApplication
2023/09/07 07:55:56.636573 [INFO] fetching environment variables...
2023/09/07 07:55:56.636595 [INFO] no go.mod file, setting GOPATH=/var/app/current
2023/09/07 07:55:56.636692 [INFO] Purge old process...
2023/09/07 07:55:56.636702 [INFO] Removing /var/app/current/ if it exists
2023/09/07 07:55:56.636711 [INFO] Renaming /var/app/staging/ to /var/app/current/
2023/09/07 07:55:56.636723 [INFO] Register application processes...
2023/09/07 07:55:56.636726 [INFO] Registering the proc: web

2023/09/07 07:55:56.636735 [INFO] Running command /bin/sh -c systemctl show -p PartOf web.service
2023/09/07 07:55:56.642189 [INFO] Running command /bin/sh -c systemctl daemon-reload
2023/09/07 07:55:56.755548 [INFO] Running command /bin/sh -c systemctl reset-failed
2023/09/07 07:55:56.759652 [INFO] Running command /bin/sh -c systemctl is-enabled eb-app.target
2023/09/07 07:55:56.763266 [INFO] Copying file /opt/elasticbeanstalk/config/private/aws-eb.target to /etc/systemd/system/eb-app.target
2023/09/07 07:55:56.766593 [INFO] Running command /bin/sh -c systemctl enable eb-app.target
2023/09/07 07:55:56.855143 [INFO] Created symlink from /etc/systemd/system/multi-user.target.wants/eb-app.target to /etc/systemd/system/eb-app.target.

2023/09/07 07:55:56.855191 [INFO] Running command /bin/sh -c systemctl start eb-app.target
2023/09/07 07:55:56.860136 [INFO] Running command /bin/sh -c systemctl enable web.service
2023/09/07 07:55:56.947217 [INFO] Created symlink from /etc/systemd/system/multi-user.target.wants/web.service to /etc/systemd/system/web.service.

2023/09/07 07:55:56.947245 [INFO] Running command /bin/sh -c systemctl show -p PartOf web.service
2023/09/07 07:55:56.952015 [INFO] Running command /bin/sh -c systemctl is-active web.service
2023/09/07 07:55:56.955391 [INFO] Running command /bin/sh -c systemctl start web.service
2023/09/07 07:55:56.975099 [INFO] Job for web.service failed because the control process exited with error code. See "systemctl status web.service" and "journalctl -xe" for details.

2023/09/07 07:55:56.975126 [ERROR] An error occurred during execution of command [app-deploy] - [FlipApplication]. Stop running the command. Error: Register application failed because the registration of proc web failed:
startProcess Failure: starting process "web" failed: Command /bin/sh -c systemctl start web.service failed with error exit status 1. Stderr:Job for web.service failed because the control process exited with error code. See "systemctl status web.service" and "journalctl -xe" for details.

2023/09/07 07:55:56.975130 [INFO] Executing cleanup logic
2023/09/07 07:55:56.975229 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment successfully generated a 'Procfile'.","timestamp":1694073354893,"severity":"INFO"},{"msg":"Instance deployment: You didn't include a 'go.mod' file in your source bundle. Elastic Beanstalk set the 'GOPATH' environment property to the application path.","timestamp":1694073356636,"severity":"INFO"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1694073356975,"severity":"ERROR"}]}]}

2023/09/07 07:55:56.975380 [INFO] Platform Engine finished execution on command: app-deploy

2023/09/07 08:13:18.644755 [INFO] Starting...
2023/09/07 08:13:18.644792 [INFO] Starting EBPlatform-PlatformEngine
2023/09/07 08:13:18.644808 [INFO] reading event message file
2023/09/07 08:13:18.644983 [INFO] Engine received EB command cfn-hup-exec

2023/09/07 08:13:18.708319 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:us-west-2:147985732667:stack/awseb-e-wpy2emnhff-stack/d095b300-4d53-11ee-a7a3-0afc390dc713 -r AWSEBAutoScalingGroup --region us-west-2
2023/09/07 08:13:19.051654 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-get-metadata -s arn:aws:cloudformation:us-west-2:147985732667:stack/awseb-e-wpy2emnhff-stack/d095b300-4d53-11ee-a7a3-0afc390dc713 -r AWSEBBeanstalkMetadata --region us-west-2
2023/09/07 08:13:19.377310 [INFO] checking whether command tail-log is applicable to this instance...
2023/09/07 08:13:19.377321 [INFO] this command is applicable to the instance, thus instance should execute command
2023/09/07 08:13:19.377324 [INFO] Engine command: (tail-log)

2023/09/07 08:13:19.377386 [INFO] Executing instruction: GetTailLogs
2023/09/07 08:13:19.377389 [INFO] Tail Logs...
2023/09/07 08:13:19.377668 [INFO] Running command /bin/sh -c tail -n 100 /var/log/web.stdout.log
2023/09/07 08:13:19.379391 [INFO] Running command /bin/sh -c tail -n 100 /var/log/eb-engine.log


/var/log/eb-hooks.log

Filter out blank shadow files

Dufflebag currently whitelists all /etc/shadow files. But in practice, a lot of these are blank and don't contain any interesting info. It would be nice to do a light parsing of these files and only upload it if there's a password hash inside.

Missing dependency Blake3

100- % of customers impacted: [100]%
Engineers- Affected Users: [Engineers / Operators / Customers]
Low- Urgency: [High / Medium / Low]

There is a dependency missing for Blake3. I'm new to go and have been unable to make it work.

Thanks for your help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.