Blank SVG file generated

[anaynayak]

Blank SVG file generated

[anaynayak]

I'm assuming you are using awscli v1.7.36. I'm able to see it correctly with a test account. Can you try it with DEBUG turned on ?

DEBUG=true bundle exec ruby lib/visualize_aws.rb -o security_groups.json -f viz.svg --color

It will print out all the nodes and edges that it detects. This should help identify the issue.

[garlandkr]

aws-cli/1.7.38 Python/2.7.6 Darwin/14.4.0

It prints all of the nodes and edges and finally the output line:

output: {:svg=>"viz.svg", :use=>"fdp"}

Here is an example from the svg file:

<!-- DEPLOYMENT&#45;SecGrp -->
<g id="node6" class="node"><title>DEPLOYMENT&#45;SecGrp</title>
<ellipse fill="none" stroke="black" cx="656601" cy="-468282" rx="98.8943" ry="18"/>
<text text-anchor="middle" x="656601" y="-468277" font-family="Times,serif" font-size="14.00">DEPLOYMENT&#45;SecGrp</text>
</g>
<!-- 1.2.3.4/32&#45;&gt;DEPLOYMENT&#45;SecGrp -->
<g id="edge5" class="edge"><title>1.2.3.4/32&#45;&gt;DEPLOYMENT&#45;SecGrp</title>
<path fill="none" stroke="#ffefdb" stroke-width="2" d="M665772,-473929C665242,-473602 657251,-468682 656638,-468304"/>
<polygon fill="#ffefdb" stroke="#ffefdb" stroke-width="2" points="656639,-468301 656629,-468299 656635,-468307 656639,-468301"/>
<text text-anchor="middle" x="661161" y="-471119" font-family="Times,serif" font-size="14.00">22/tcp,8080/tcp</text>
</g>

This is the JSON file for it from aws cli:

        {
            "IpPermissionsEgress": [], 
            "Description": "Used by the deployment server.", 
            "IpPermissions": [
                {
                    "PrefixListIds": [], 
                    "FromPort": 22, 
                    "IpRanges": [
                        {
                            "CidrIp": "5.6.7.8/32"
                        }, 
                        {
                            "CidrIp": "1.2.3.4/32"
                        }
                    ], 
                    "ToPort": 22, 
                    "IpProtocol": "tcp", 
                    "UserIdGroupPairs": []
                }, 
                {
                    "PrefixListIds": [], 
                    "FromPort": 8080, 
                    "IpRanges": [
                        {
                            "CidrIp": "5.6.7.8/32"
                        }, 
                        {
                            "CidrIp": "1.2.3.4/32"
                        }
                    ], 
                    "ToPort": 8080, 
                    "IpProtocol": "tcp", 
                    "UserIdGroupPairs": []
                }
            ], 
            "GroupName": "DEPLOYMENT-SecGrp", 
            "OwnerId": "xxxxxxxxxxxx", 
            "GroupId": "sg-xxxxxxxx"
        },

[anaynayak]

Oh so it generates the svg file with all the nodes as well ? In that case I suspect that the generated svg file is quite large and probably the viewer renders it at 100% ?

This might sound stupid but can you try zooming out? On some of the projects where I've tried it, it has generated very large svgs. Alternatively you can export it to another format .png? and try if that works.

If you have a svg file that is large enough, I suspect that it is already generated correctly.

You can also tweak the mode from fdp to dot or sfdp and check if it renders it differently.
in lib/visualize_aws.rb

g.output(extension[1..-1].to_sym => output_file, :use => 'fdp') #replace fdp with dot

[garlandkr]

Zooming didn't show anything else. I tried using PNG output instead and the Preview app in OSX said:

The file “security_groups.png” could not be opened because it is empty.

I've also tried changing fdp to dot in visualize_aws.rb and the application seems to be running endlessly. I am checking for file size increase in the output file using another shell but the file is not changing. When I open that .svg file it just shows a small black square in the upper left corner. Changing it to sfdp renders a thin line in the upper left corner and gives an error in the shell:

/Library/Ruby/Gems/2.0.0/gems/ruby-graphviz-1.2.1/lib/graphviz/utils.rb:61:in `output_from_command': Error from ["/usr/local/bin/sfdp", "-q1", "-Tsvg", "-oviz.svg", "/var/folders/r7/zw1_svks0qqdtzz85mzh6yn10zzh5b/T/graphviz.rb20150717-79056-1uc5m94"]: (RuntimeError)
Error: remove_overlap: Graphviz not built with triangulation library
    from /Library/Ruby/Gems/2.0.0/gems/ruby-graphviz-1.2.1/lib/graphviz.rb:600:in `output'
    from /Users/kgarland/code/aws-security-viz/lib/graph.rb:29:in `output'
    from lib/visualize_aws.rb:38:in `render'
    from lib/visualize_aws.rb:17:in `unleash'
    from lib/visualize_aws.rb:55:in `<main>'

[anaynayak]

This seems to be a bug :(

Would it be possible for you to send across the json file to me @ [email protected] ? Alternatively, I've just added code to obfuscate the node names and labels.

If you can send me the console snippet for debug that should do as well.

DEBUG=true OBFUSCATE=true bundle exec ruby lib/visualize_aws.rb -o security_groups.json -f viz.svg --color

[garlandkr]

Ok, I've sent the obfuscated svg file to the e-mail address listed.

[anaynayak]

It seems that Graphviz has issues rendering a svg with 4000+ edges. I was able to render it using sfdp and graphviz.

If you get the "Graphviz not built with triangulation library" error message you would have to reinstall graphviz with the following options on Mac

brew install graphviz --with-gts

Can you let me know if that works for you ?

[anaynayak]

Also, I'm not sure if the rendered image with real (non-obfuscated) data is easily readable. I've been thinking of how to provide a meaningful visualization with bigger data.

One option was to provide a mechanism to combine ip ranges using groups (undocumented)

Does that apply to your use case as well ?

[garlandkr]

Ok, it renders now but there are way too many nodes for it to be readable.

Using groups would be a great benefit. In my topology some subnets across VPCs can be grouped as well. For example 10.20.20.0/24 and 172.16.0.0/16 could both be part of a group that uses VPC peering for specific traffic.

[anaynayak]

Does the current approach for defining groups work for the usecase you specified ? I'm not sure if that makes the nodes visible or are there other things that we still need to do to get that working.

Few ideas:

1. Provide the option to filter out nodes based on a pattern. This should help remove development/test groups.
2. Provide a simpler way (yml?) of providing group definitions. Something that doesn't involve changing code.
3. Provide a web export option which can then be interactive.

[garlandkr]

The option to filter nodes based on a pattern would be great, this would reduce a lot of the information being rendered.

I like the idea of an interactive option so that clicking on a group/node/etc name turns the other nodes/edges/lines grey, or were you thinking of something that gives the option to enable/disable items directly in the web page?

Let me give an example of our naming convention.

EC2 Instances:
{{ environment }}-{{ group }}-{{ service_name }}

Load Balancers:
{{ environment }}-{{ group }}-{{ service_name }}-{{ external/internal }}-elb

So we could have a list of the following security groups for a single application in a single environment:

dev-group1-test_java_app
dev-group1-test_java_app-internal-elb
dev-group1-test_java_app-external-elb

Likely they will all be connected to each other, they might also share a common aws-security-viz group called "Internal" where dev-group1-test_java_app and dev-group1-test_java_app-internal-elb both have internal ip addresses allowed.

[anaynayak]

Cool. I'll probably externalize the config to a yml file so that things like filter patterns, grouping etc can be defined at one place.

For the patterns, I'm thinking of using them to define exclusion criteria so that everything else is included by default. This also takes care of security groups which are shared.

For the web view, I was thinking of providing a mechanism to generate a preconfigured web view using something like jquery.graphviz.svg.

See example : https://cdn.rawgit.com/mountainstorm/jquery.graphviz.svg/master/demo.html

[anaynayak]

Hello,

I've externalized the configuration to yml. You should now be able to define exclusion patterns and grouping via the yml. Rename https://github.com/anaynayak/aws-security-viz/blob/master/opts.yml.sample to opts.yml and customize as required.

I'm still working on defining the web view with jquery.graphviz.svg. Tracking this explicitly as part of #15

Let me know if the exclusion, grouping works for you.

[garlandkr]

Amazing, you are really quick.

This works well, I was able to get a very readable output for my different environments.