ronin-rb/ronin-support

A support library for Ronin. Like activesupport, but for hacking!

ronin-support

• Source
• Issues
• Documentation
• Discord | Mastodon

Description

ronin-support is a support library for Ronin. ronin-support provides many Core Extensions to Ruby's built-in classes as well as its own Classes/Modules. ronin-support can be used by other Ruby libraries, tools, or scripts.

tl;dr It's like pwntools combined with activesupport.

ronin-support is part of the ronin-rb project, a Ruby toolkit for security research and development.

Features

• Provides user-friendly APIs for:
  • Bit-flipping
  • Hexdump / unhexdump data.
  • Packing/unpacking binary data:
    • C types
    • Buffers
    • IO streams
    • Stacks
    • Strings
    • Arrays
    • Structs
    • Unions
  • Encoding data:
    • Base16
    • Base32
    • Base64
    • C strings
    • Hex
    • HTML
    • HTTP
    • JavaScript
    • PowerShell
    • Punycode
    • Quoted-printable
    • Ruby strings
    • Shell
    • SQL
    • URI
    • UUencoding
    • XML
  • Reading/writing compressed data:
    • Zlib
    • Gzip
  • Reading/writing archive files:
    • Tar
    • Zip
  • Cryptography:
    • RSA
    • DSA
    • DH
    • EC
    • HMAC
    • Ciphers
    • X509 certificates
  • Networking:
    • DNS
    • UNIX
    • TCP
    • UDP
    • SSL / TLS
    • FTP
    • SMTP / ESMTP
    • POP3
    • IMAP
    • Telnet
    • HTTP / HTTPS
    • Raw packets
    • ASNs
    • IP addresses
    • IP ranges
    • TLDs
    • Public Suffix List
    • Host names
    • Domain names
  • Working with text:
    • Generating typos.
    • Generating homoglyphs.
    • Regexs for matching/extracting common types of data.
• Adds additional methods to many of Ruby's core classes.
• Small memory footprint (~46Kb).
• Has 96% documentation coverage.
• Has 93% test coverage.

Synopsis

$ irb -r ronin/support
irb(main):001:0> "hello world".base64_encode
=> "aGVsbG8gd29ybGQ=\n"
irb(main):002:0> "aGVsbG8gd29ybGQ=\n".base64_decode
=> "hello world"

Examples

require 'ronin/support'
include Ronin::Support

string = "hello world"
puts string.base64_encode

data = "aGVsbG8gd29ybGQ=\n"
puts data.base64_decode

For more examples of the convenience methods provided by ronin-support, please see the API documentation.

Requirements

• Ruby >= 3.0.0
• chars ~> 0.3, >= 0.3.2
• hexdump ~> 1.0
• combinatorics ~> 0.4
• addressable ~> 2.0
• uri-query_params ~> 0.8

Install

$ gem install ronin-support

Gemfile

gem 'ronin-support', '~> 0.5'

Development

1. Fork It!
2. Clone It!
3. cd ronin-support
4. bundle install
5. git checkout -b my_feature
6. Code It!
7. bundle exec rake spec
8. git push origin my_feature

License

Copyright (c) 2006-2023 Hal Brodigan (postmodern.mod3 at gmail.com)

ronin-support is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

ronin-support is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with ronin-support. If not, see https://www.gnu.org/licenses/.

ronin-support's Issues

Rename udp_single_server to udp_recv

Rename udp_single_server to udp_recv. Deprecate udp_single_server as well.

Add String#encrypt, String#decrypt, File.encrypt, File.decrypt

Add cryptographic helper methods for encrypting/decrypting data using OpenSSL. Use aes_pipe.rb as a starting point.

Add String.parse

There should be a method which parses single and double-quoted Strings.

Add udp_single_server

Add the udp_single_server method to Ronin::Network::UDP, in the style of Ronin::Network::TCP#tcp_single_server. This method should use Ronin::Network::UDP#udp_server_session, but call UDPSocket#recvfrom to accept a single message and the server address.

dns_lookup and dns_lookup_all will not work on JRuby 1.6 in 1.9 mode

The dns_lookup and dns_lookup_all methods will not work on JRuby 1.6 in 1.9 mode, due to JRuby bug #5647.

Workaround

Run JRuby 1.6 in it's default 1.8 mode.

Add a hash identification method

There should be a hash identification method that checks length/charset. It could be added to String or maybe Digest? Unsure on name or where in ronin-support it should be added to.

cc @nullthreat

#pack / #unpack methods incorrectly treat :net byte order as :little

Network byte order is defined as big endian.

Add Ronin::Network::Proxy

Add a Ronin::Network::Proxy class for analyzing and manipulating TCP(SSL)/UDP protocols. This Proxy class would probably heavily use IO.select (instead of EventMachine or nio4r) to watch for IO events, and have callbacks for inspect/manipulating data.

Map String#unpack to Binary::Template#unpack

If String#unpack is given Symbol arguments, use Binary::Template#unpack.

data.unpack(:uint32, :uint32, :string)

Add tcp_open? and udp_open? to TCP/UDP Mixins

Add missing tcp_open? and udp_open? methods to TCP/UDP Mixins.

Binary::Hexdump::Parser should parse floats

The Hexdump Parser should support parsing floats.

Rescue exceptions from within tcp_banner / udp_banner

Rescue SystemCallError and SocketError exceptions within tcp_banner and udp_banner. If we could not connect to the port or the recv timed out, nil shall be returned.

Add String#fuzz

Add a fuzz method to the String class. This method will enumerate over possible mutations of a String. Can probably backport most of the code from sophsec/wordlist.

Examples

# Replace ":" with "\0"
"Content-Type: text/html".fuzz(':' => "\0")

# Replace ":" and the ending with "\0"
"Content-Type: text/html".fuzz(/[:$]/ => "\0")

# Replace ":" with "\n", "\r\n", "\0" then "::"
"Content-Type: text/html".fuzz(':' => ["\n", "\r\n", "\0", "::"])

# Replace ":" with bytes
"Content-Type: text/html".fuzz(':' => (0..0x10))

Go 1.9 mode only

MRI 1.8.7 has been EOLed. 1.9.x provides many additional convenience methods (such as File.write / File.binwrite). We could require ruby_version >= 1.9.1 or add backports as a dependency.

• Ubuntu 12 now provides Ruby 1.9.2 packages.
• Fedora 17 now provides Ruby 1.9.3 packages.
• OSX still provides Ruby 1.8.7, although most OSX users prefer using RVM or rbenv.
• JRuby 1.7.0 will use 1.9 mode as the default mode.

Allow Array#pack to accept Binary::Template types.

Array#pack should also accept Binary::Template types, like String#unpack.

[0x42, "hello"].pack(:uint32_le, :string)

No longer hook puts, write, etc in Ronin::UI::Output::Helpers

Hooking the puts and write methods causes issues with RSpec and Rails. It's not safe to override such widely used IO methods.

Add Binary::Struct

Add a simple Binary Struct, similar to FFI::Struct but using Binary::Packer under the hood (see issue #18).

Example

class Packet < Binary::Struct

  layout :length, :uint,
         :data, [:uchar, 100]

  def length
    @length || data.length 
  end

end

pkt = Packet.unpack(buffer)
pkt.length # => 5
pkt.data   # => "hello"

pkt = Packet.new
pkt.data = "hello"
pkt.pack
# => "\x00\x00\x00\x05hello\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"

String#unhexdump should handle alternate endianness.

Just noticed that readelf -x outputs a hexdump of hexadecimal integers, but in big-endian encoding.

String#unhexdump was originally written with the od utility in mind, which outputs little-endian packed integers.

Add missing ssl methods to Network::Mixins::SSL

Add missing SSL methods to Ronin::Network::Mixins::SSL.

• ssl_open?
• ssl_connect_and_send
• ssl_send
• ssl_banner
• ssl_server_loop
• ssl_accept

Add missing ssl network methods

Ronin::Network::SSL is missing the following methods:

• ssl_open? (just use tcp_open?)
• ssl_connect_and_send
• ssl_banner
• ssl_send
• ssl_server
• ssl_server_session
• ssl_server_loop
• ssl_accept

Add Binary::Template

Add a translation layer between binary types (:uint, :float, etc) and Array#pack codes.

Example

Template::TYPES[uint32_be] # => "N"

template = Template.new(:uint, [:uchar, 20])
template.pack(1234, "hello")
# => "\x00\x00\x04\xD2hello\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
template.unpack(_)
# => [1234, "hello"]

Should Binary::Hexdump::Parser return Integers/Floats, bytes or the raw String?

Should the Binary::Hexdump::Parser return Integers/Floats, the parsed bytes or the full raw String? Currently the Binary::Hexdump::Parser returns an Array of bytes.

Add Network helper methods and a Mixin for FTP

Ruby has built-in support for the FTP protocol (Net::FTP). ronin-support needs Network helper methods and a Mixin, similar to Network::HTTP and Network::Mixins::HTTP.

Rename Ronin::UI::Output to Ronin::Printing

Ronin::Printing or Ronin::UI::Printing sounds better.

Add basic documentation to the Fuzzing methods.

The fuzzing methods needs basic YARD documentation.

Support comma seperated octets in IPAddr.each.

Support nmap style comma separated octets in IPAddr.each:

IPAddr.each('192.168.1.5,35,110')

Should enumerate over:

1. 192.168.1.5
2. 192.168.1.35
3. 192.168.1.110

Add a Network::SOCKS mixin

Ruby provides the SOCKSSocket class for connecting to SOCKS5 proxies. Ronin should provide a Network mixin for this.

Add String#xml_escape / String#xml_unescape

Add String#xml_escape / String#xml_unescape methods that are aliases to their html_ equivalents.

Add Network helper methods for working with UNIXSockets

Network helper methods and a Mixin should be added for working with UNIXSocket and UNIXServer, similar to the TCP/UDP Network helpers.

Consolidate network helper methods, deprecate old ones.

The number of TCP/UDP/UNIX helper methods has steadily increased. I think it's time to combine some of these specialized methods. Also, should this change occur in 0.5.0 or 0.6.0?

1. Combine tcp_connect and tcp_session into tcp_open. When not given a block, the socket will be returned. When given a block, the socket will be yielded, then closed. Also this method could be called tcp_socket or simply tcp.
2. Combine tcp_server and tcp_server_session into tcp_listen. When not given a block, the socket will be returned. When given a block, the socket will be yielded, then closed.

Add specs for Ronin::Fuzzing::Mutator and String#mutate

String#mutate in ronin/fuzzing/extensions/string does not have specs.

Add String#http_escape and String#http_unescape methods

It is awkward to use uri_escape / uri_unescape methods when encoding HTTP head values (Cookie). There should be http_escape / http_unescape methods, that either call URI.escape or CGI.escape.

Decompose String#unhexdump into a Hexump::Parser class

Decompose the rather large String#unhexdump method into a Hexdump::Parser class.

Hash identification method?

Should ronin-support have a Hash identification method. This method would accept a String, and determine possible hashing algorithms that may have generated it.

Add digest methods for rmd160

Add digest methods to String and File for Digest::RMD160.

Add TCP/UDP helper methods to test if a remote port is open

There should be TCP/UDP network helper methods that test if a remote port is open. These methods should also accept a timeout argument.

Add Float#pack

There should be a Float#pack method to match Integer#pack.

Rename tcp_single_server to tcp_accept

Rename tcp_single_server to tcp_accept. Deprecate tcp_single_server as well.

Add Ronin::Network::Mixins::UNIX

Add a Network Mixin for Ronin::Network::UNIX.

Move js_escape / js_unescape methods into their own js/ directory

The js_escape / js_unescape methods should not be in the html/ directory.

Add convenience methods for Digest::HMAC

There should be methods similar to String#sha1, but for Digest::HMAC.

Map Integer#pack to Binary::Template#pack

Integer#pack should use Binary::Template#pack.

v0.5.0.rc2 ronin wordlist not producing a mutated wordlist

Consider the following from the command line:

ronin wordlist -v -i wlmutate -m "{:o=>'4'}"
bar
baz
foo

where the contents of wlmutate are

foo
bar
baz

0.5.0.rc1 Ronin::Wordlist#save failure

require 'ronin/wordlist'
words = %w[foo bar baz]
wl = Ronin::Wordlist.new(words, 'o' => ['0', '()'])
wl.save('wlmutate')

results in:

<path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/fuzzing/extensions/string.rb:339:in `block in mutate': uninitialized constant String::StringScanner (NameError)
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/fuzzing/extensions/string.rb:318:in `each'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/fuzzing/extensions/string.rb:318:in `mutate'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:232:in `block in each'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:206:in `each'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:206:in `each_word'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:227:in `each'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:275:in `block in save'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:274:in `open'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc1/lib/ronin/wordlist.rb:274:in `save'

Add tcp_server_loop and udp_server_loop

Add the Ronin::Network::TCP#tcp_server_loop and Ronin::Network::UDP#udp_server_loop methods in the vain of Socket.tcp_server_loop and Socket.udp_server_loop.

Ronin::Network::UDP::Proxy.start issue with socket.recv

Consider:

require 'ronin/network/udp/proxy'
require 'hexdump'

Ronin::Network::UDP::Proxy.start(:port => 1194, :server => ['meow.com', 1194]) do |proxy|
  address = lambda { |socket|
    addrinfo = socket.peeraddr

    "#{addrinfo[3]}:#{addrinfo[1]}"
  }
  hex = Hexdump::Dumper.new

  proxy.on_client_data do |client,server,data|
    puts "#{address[client]} -> #{proxy}"
    hex.dump(data)
  end

  proxy.on_server_data do |client,server,data|
    proxy.close! if data =~ /QUIT/

    puts "#{address[client]} <- #{proxy}"
    hex.dump(data)
  end

end

results in:

$ ruby udp_proxy.rb 
<path>/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc2/lib/ronin/network/udp/proxy.rb:118:in `recv': undefined method `recv' for nil:NilClass (NoMethodError)
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc2/lib/ronin/network/udp/proxy.rb:67:in `poll'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc2/lib/ronin/network/proxy.rb:190:in `listen'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc2/lib/ronin/network/proxy.rb:155:in `start'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-support-0.5.0.rc2/lib/ronin/network/proxy.rb:142:in `start'
    from udp_proxy.rb:4:in `<main>'

Split js_* convenience methods out of formatting/extensions/html

js_* convenience methods should be in their own directory. @mephux will be handling this.

v0.5.0.rc2 ronin wordlist incomplete argument sanity checking

I don't think this is very important, but there is not great error handling for the arguments. Assume I use the following command:

ronin wordlist -v -i wlmutate -m "{o=>'4'}" 
<path>/.rvm/gems/ruby-1.9.3-p194/gems/parameters-0.4.2/lib/parameters/options.rb:159:in `merge!': can't convert nil into Hash (TypeError)
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/parameters-0.4.2/lib/parameters/options.rb:159:in `block in define'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1391:in `call'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1391:in `block in parse_in_order'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1347:in `catch'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1347:in `parse_in_order'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1341:in `order!'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1432:in `permute!'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1453:in `parse!'
    from <path>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/optparse.rb:1443:in `parse'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-1.4.1/lib/ronin/ui/cli/command.rb:211:in `start'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-1.4.1/lib/ronin/ui/cli/command.rb:166:in `start'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-1.4.1/lib/ronin/ui/cli/cli.rb:137:in `start'
    from <path>/.rvm/gems/ruby-1.9.3-p194/gems/ronin-1.4.1/bin/ronin:25:in `<top (required)>'
    from <path>/.rvm/gems/ruby-1.9.3-p194/bin/ronin:23:in `load'
    from <path>/.rvm/gems/ruby-1.9.3-p194/bin/ronin:23:in `<main>'

This should probably give an error message that describes that I should be using a specific ruby hash syntax

Move #unhexdump methods into ronin/extensions/

Move the #unhexdump methods out of ronin/formatting/extensions/binary/ and into ronin/extensions/, since they are not formatting methods.

Rename unix_single_server to unix_read

Rename unix_single_server to unix_read. Deprecate unix_single_server as well.