mirage/ocaml-ipaddr

A library for manipulation of IP (and MAC) address representations

ipaddr: IP and MAC address manipulation

A library for manipulation of IP and MAC address representations.

Features:

• ounit2-based tests
• IPv4 and IPv6 support
• IPv4 and IPv6 CIDR prefix support
• IPv4 and IPv6 CIDR-scoped address support
• Ipaddr.V4 and Ipaddr.V4.Prefix modules are Map.OrderedType
• Ipaddr.V6 and Ipaddr.V6.Prefix modules are Map.OrderedType
• Ipaddr and Ipaddr.Prefix modules are Map.OrderedType
• Ipaddr_unix in findlib subpackage ipaddr.unix provides compatibility with the standard library Unix module
• Ipaddr_top in findlib subpackage ipaddr.top provides top-level pretty printers
• IP address scope classification
• IPv4-mapped addresses in IPv6 (::ffff:0:0/96) are an embedding of IPv4
• MAC-48 (Ethernet) address support
• Macaddr is a Map.OrderedType
• All types have sexplib serializers/deserializers optionally via the Ipaddr_sexp and Macaddr_sexp libraries.

Usage

There are the following opam packages included:

• ipaddr: the Ipaddr and associated modules
• ipaddr-sexp
• ipaddr-cstruct
• macaddr: the Macaddr and associated modules.
• macaddr-sexp
• macaddr-cstruct

There are the following ocamlfind libraries included as part of this repository, included as part of the respective opam packages.

• ipaddr: The Ipaddr module for IPv4/6 manipulation.
• ipaddr.top: Toplevel printers for Ipaddr.
• ipaddr-cstruct: The Ipaddr_cstruct module
• macaddr: The Macaddr module for MAC address manipulation.
• macaddr.top: Toplevel printers for Macaddr.
• macaddr-cstruct: The Macaddr_cstruct module
• ipaddr-sexp: S-expression converters for Ipaddr.
• macaddr-sexp: S-expression converters for Macaddr.

Installation and development

The packages are released to the opam-repository. An opam install ipaddr (or any other above mentioned package) will install it. If you want to install the latest development commit, opam pin add ipaddr --dev will do this.

A local build, after a git clone can be done with dune build, a dune runtest compiles and executes the testsuite. If dependencies are missing, opam install (-t) --deps-only . in the cloned directory will install them.

The auto-formatter ocamlformat is used, please execute dune build @fmt --auto-promote before submitting a pull request.

Contact

• Issues: https://github.com/mirage/ocaml-ipaddr/issues
• E-mail: [email protected]
• API Documentation: http://docs.mirage.io/ipaddr/
• Discussion: https://discuss.ocaml.org with the mirageos tag.

ocaml-ipaddr's Issues

ppx_deriving friendly

For @whitequark fanclub.

Prefix.of_netmask_address

Please file an issue with ocaml-ipaddr if you think something like a Prefix.of_netmask_address : addr -> addr -> t would be generally useful.

It would be best to obtain a prefix from what getifaddrs returns.

`V6.to_int64` is broken if the 2nd or 4th 32-bit integer in an IPv6 address is greater than `2^31-1`

I noticed this when trying to encode the IP address 2a01:4f9:c011:87ad:0:0:0:0 in ocaml-dns, but decoding it resulted in ffff:ffff:c011:b100::1.

to_int64 is defined as:

  let to_int64 (a, b, c, d) =
    Int64.
      ( logor (shift_left (of_int32 a) 32) (of_int32 b),
        logor (shift_left (of_int32 c) 32) (of_int32 d) )

As b is a signed 32 bit integer, if b is negative Int64.of_int32 b will create a negative 64-bit integer.

That is, if we have the IP address 0:0:8000:0:0:0:0:0, with b=1000000000000000 0000000000000000, then Int64.of_int32 b will equal 1111111111111111 1111111111111111 1000000000000000 0000000000000000 instead of 0000000000000000 0000000000000000 1000000000000000 0000000000000000.

The same is true of Int64.of_int32 d.

This isn't a problem for a and b as they are shifted left 32 bits anyway.

My proposed fix is to change to_int64 to:

  let to_int64 (a, b, c, d) =
    let of_uint32 i = if i < 0l then
      let (+) = Int64.add in
      Int64.of_int32 i + Int64.shift_left 1L 32
      else Int64.of_int32 i
    in
    Int64.
      ( logor (shift_left (of_int32 a) 32) (of_uint32 b),
        logor (shift_left (of_int32 c) 32) (of_uint32 d) )

Which is similar to what is used to encode the altitude in the LOC record in ocaml-dns.

If anyone has a better proposal to fix this, please let me know. Otherwise I'll create a PR with this fix.

API inconsistency: no Macaddr.pp_hum

we have a to_string, but we should provide a pretty printer, shouldn't we?

safe_string

This should be safe_string compatible.

Short CIDR notation

Hi,

I'm trying to parse a configuration file (in the pf firewall format) that has entries like:

utop # Ipaddr.V4.Prefix.of_string "172.16/12";;
- : Ipaddr.V4.Prefix.t option = None
utop # Ipaddr.V4.Prefix.of_string "10/8";;
- : Ipaddr.V4.Prefix.t option = None

I can work around that by using something like

let expand_ipv4 cidr =
  let [prefix;mask] = String.split_on_char '/' cidr in
  let provided_octets = List.length (String.split_on_char '.' prefix) in
  let padding = String.init ((4 - provided_octets)*2)
    (function | i when i mod 2 = 0 -> '.'
              | _ -> '0')
  in prefix ^ padding ^ "/" ^ mask

utop # Ipaddr.Prefix.of_string "10.0/8";;
- : Ipaddr.Prefix.t option = None

utop # Ipaddr.Prefix.of_string (expand_ipv4 "10.0/8") ;;
- : Ipaddr.Prefix.t option = Some 10.0.0.0/8

It would be very convenient for me if this type of short notation was handled by the ipaddr upstream, but I wanted to hear your thoughts first before writing a PR.

Ping @djs55

missing dependency

2.8.0 dropped a dependency on sexplib, but as far as I can see there is still (automatically generated) code referencing Sexplib.Sexp.t et al.: the jbuild file contains such a dependency, but the opam file does not.

More IPv6 Multicast Modeling

Bitfields, scopes, checks. Solicited nodes?

Add multicast MAC conversions from RFC 1112

See mirage/mirage-tcpip#81

  (* RFC 1112: 01-00-5E-00-00-00 ORed with lower 23 bits of the ip address *)
+    let mac_of_multicast ip =
+      let ipb = Ipaddr.V4.to_bytes ip in
+      let macb = String.create 6 in
+      macb.[0] <- Char.chr 0x01;
+      macb.[1] <- Char.chr 0x00;
+      macb.[2] <- Char.chr 0x5E;
+      macb.[3] <- Char.chr ((Char.code ipb.[1]) land 0x7F);
+      macb.[4] <- ipb.[2];
+      macb.[5] <- ipb.[3];
+      Macaddr.of_bytes_exn macb

Redesign parsing/printing functions for various flavors

3.0 should adopt a consistent naming scheme for parsing and printing functions with 4 axes of behavior (type, byte-rep/ASCII-rep, exceptions, offsets). Some exceptions for some functions (like Parse_error from of_cstruct_exn should probably be removed. See #33, #35, #36.

Add sexp converters for ipaddr

Makes it easier to embed in Core/Async datastructures

Is Dune broken or am I stupid?

:~/ocaml/ipaddr$ topkg build
      ocamlc lib/.ipaddr.objs/macaddr.{cmi,cmti} (exit 2)
(cd _build/default && /home/user/.opam/4.04.2+fPIC/bin/ocamlc.opt -w @a-4-29-40-41-42-44-45-48-58-59-60-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -bin-annot -I lib/.ipaddr.objs -I /home/user/.opam/4.04.2+fPIC/lib/sexplib -I /home/user/.opam/4.04.2+fPIC/lib/sexplib/0 -no-alias-deps -o lib/.ipaddr.objs/macaddr.cmi -c -intf lib/macaddr.pp.mli)
File "lib/macaddr.mli", line 27, characters 0-27:
Warning 32: unused value compare.
File "lib/macaddr.mli", line 1:
Error: Some fatal warnings were triggered (1 occurrences)
pkg.ml: [ERROR] cmd ['jbuilder' 'build' '--root' '.' '--dev']: exited with 1

what am I doing wrong?

docstring comment looks wrong

Refers to old behaviour where an option type was returned I think.

ocaml-ipaddr/lib/macaddr.mli

Line 44 in 3759a96

(** Same as above but returns an option type instead of raising an

bigstring support

Both shared and copied.

bin_prot

Would it be possible to add bin_prot along with sexp (i.e. replacing "with sexp" by "with sexp, bin_io") ?
Or at least to make it available if the user already has core installed.

Clarify bytestring conversion docstrings

We currently have this:

  (** {3 Bytestring conversion} *)

  (** [of_bytes_exn ipv4_octets] is the address represented
      by [ipv4_octets]. Raises [Parse_error] if [ipv4_octets] is not a
      valid representation of an IPv4 address. *)
  val of_bytes_exn : string -> t

  (** Same as [of_bytes_exn] but returns an option type instead of raising
      an exception. *)
  val of_bytes : string -> t option

  (** Same as [of_bytes_exn] but take an extra paramenter, the offset into
      the bytes for reading. *)
  val of_bytes_raw : string -> int -> t

From reading the docstring it was unclear to me that

if [ipv4_octets] is not a valid representation of an IPv4 address

actually means (which AFAICT is the only failure mode?):

if [ipv4_octets] is not exactly 4 bytes long.

It is also misleading that of_bytes_raw suggests that it is the same as of_bytes_exn; as evidenced below, of_bytes_raw does not impose an upper bound on the string length:

  let of_bytes_raw bs o =
    make
      (Char.code bs.[0 + o])
      (Char.code bs.[1 + o])
      (Char.code bs.[2 + o])
      (Char.code bs.[3 + o])

  let of_bytes_exn bs =
    let len = String.length bs in
    if len > 4 then raise (too_much bs);
    if len < 4 then raise (need_more bs);
    of_bytes_raw bs 0

Not overly important I guess, but this tripped me during a code review.

(Modified) EUI64 Support

MAC address relations, field processing, IPv6 address embedding

Testing membership in V4 default route

It seems like Ipaddr.V4.Prefix.mem yields an incorrect result, when testing the membership of an IPv4 address in the default route 0.0.0.0/0.

# Ipaddr.V4.Prefix.mem (Ipaddr.V4.of_string_exn "192.168.0.1")
    (Ipaddr.V4.Prefix.of_string_exn "0.0.0.0/0")
- : bool = false

The issue seems to be that the conversion of some IP addresses result in a negative number due to the 32 bit signed integers. 192.168.0.1 is converted into -1062731775l.

In the mem function, both values are shifted by 32 bits in that case and are then compared.

# Int32.shift_right_logical (-1062731775l) 32
- : int32 = -1062731775l

# Int32.shift_right_logical 0l 32
- : int32 = 0l

All other functions of the module, except for V4.Prefix.subset (which calls mem), seem to behave correctly.

I've just created this issue to track the problem. I'm still trying to come up with a better idea than just returning true for a prefix length of 0.

IPv4/IPv6 Translator Support (RFC 6052)

There are probably some utility functions we could provide.

See http://tools.ietf.org/html/rfc6052

Handle RFC 4007: IPv6 zone indices for link-local scoped addresses

Source: https://en.wikipedia.org/wiki/IPv6_address#Scoped_literal_IPv6_addresses

I had a use for this in SSH forwarding, and wondered if the OCaml SSH stack would be able to support this at some point, so I tried:

utop # Ipaddr.V6.of_string "fe80::1234:5678:9012%em0";;
- : Ipaddr.V6.t option = None

Relevant RFC: RFC 4007 section 6

An implementation SHOULD support at least numerical indices that are
non-negative decimal integers as <zone_id>. The default zone index,
which should typically be 0 (see Section 6), is included in the
integers. When <zone_id> is the default, the delimiter characters
"%" and <zone_id> can be omitted. Similarly, if a textual
representation of an IPv6 address is given without a zone index, it
should be interpreted as

%, where
is the default zone index of the scope that has.

So we should support things like fe80::1%0.

RFC 4007 continues:

An implementation MAY support other kinds of non-null strings as
<zone_id>. However, the strings must not conflict with the delimiter
character. The precise format and semantics of additional strings is
implementation dependent.

In Linux and BSD it seems like the IP stacks (or socket libraries or whatever) accept strings to specify the network interface, like %wifi0, or at least provides a way to resolve the numeric ID from the string automatically. This seems more useful than the numerical ID to me, but that is not compatible with Windows (where you would have to look up the interface).

FreeBSD also accepts stuff like fe80::123:4567:8910:1112%lagg0.1181 (for if lagg0 vlan 1181).

(Thanks a lot to @tykling for helping explain the use of % and for providing FreeBSD examples)

1. I'm not sure what the best way to expose these zone ID hints would be, but for starters I think it would be nice to simply strip trailing %..* and parse the vanilla IPv6 address.

2. Reading wikipedia, addresses this range should support zone indices:

fe80::/10

3. Reading RFC 3513 section 2.7 I interpret it to extend to the "transient local" multicast addresses in this range too (please correct me if I got the CIDR prefix wrong):

ff12::/16

Some test cases:

fe80::1ff:fe23:4567:890a%0
fe80::1%em0
fe80::1ff:fe23:4567:890a%123
ff12::ff00:fe23:4567:890a%10
fe80::123:4567:8910:1112%lagg0.1181

can't install [email protected] because of bug in [email protected]

OS: windows 10
package manager: esy

log file: https://gist.github.com/Et7f3/ae5b0be2ed0331947eb11c55d06313ed#file-failed_log-txt
minimal repro: https://gist.github.com/Et7f3/ae5b0be2ed0331947eb11c55d06313ed#file-package-json

step to repdroduce:

• install esy
• download package.json in empty folder
• open cmd.exe
• travel to the folder that contain only package.json
• type esy

some note:
you can see the graph of dependencies in esy.lock/index.json
the first time it will build ocaml compiler so it is availabel inside esy: but once it is build: it will be faster
some warning are already fixed in a PR that is not merged (but esy don't care about warning) so it is the only important section I think

    File "lib/ipaddr.ml", line 233, characters 4-24:
    233 |     Macaddr.of_bytes_exn (Bytes.to_string macb)
              ^^^^^^^^^^^^^^^^^^^^
    Error: Unbound value Macaddr.of_bytes_exn
    Hint: Did you mean of_octets_exn?

Hello,
I try for a project ot install cohttp-lwt-unix but I get the error above. I have added some information that might be helpful. If you want more info/test I am here.

RFC 4007 Scoped Address support

Should be able to parse and represent local addresses with zone identifiers.

See http://tools.ietf.org/search/rfc4007

Use 16-byte string for internal representation of IPv6 addresses

A bytestring will be cheaper than a quad of int32s (2 indirections, 5 extra boxes).

netmask functions for V6.Prefix

I need those as well.

This requires Dune >= 1.6

This uses the latest version of Dune (1.6) but doesn't announce it in the opam file - leading to build failures. it would be good to either not demand it in dune-project or to announce it in the opam file.

Confusing to use bytes now that it is an OCaml type

This library is using bytes vs string as a way to descriminate between human readable strings and machine representation of IP addresses. Now that there is a byte type in OCaml, this is very confusing. The interface should be probably changed (version 3) to something else.

Support RFC 6598, Carrier-Grade NAT Shared Address Space block

See http://tools.ietf.org/html/rfc6598.

test B128 fails on i586

ocaml-4.10.0 on 32-bit i586 ALT Linux

[builder@localhost ocaml-ipaddr-5.0.0]$ dune runtest
test_ipaddr_b128 alias lib_test/runtest (exit 1)
(cd _build/default/lib_test && ./test_ipaddr_b128.exe)
F
==============================================================================
Failure: Test B128 module:0:shift_right

::ffff:ffff >> -8
expected: Ok :: but got: Ok ::ff
------------------------------------------------------------------------------
Ran: 1 tests in: 0.00 seconds.
FAILED: Cases: 1 Tried: 1 Errors: 0 Failures: 1 Skip:  0 Todo: 0 Timeouts: 0.
 test_ipaddr alias lib_test/runtest (exit 1)
(cd _build/default/lib_test && ./test_ipaddr.exe)
........................F
==============================================================================
Failure: Test V4:24:prefix_first_last

last 169.254.169.254/32
expected: 169.254.169.254 but got: 255.255.255.255
------------------------------------------------------------------------------
Ran: 25 tests in: 0.01 seconds.
FAILED: Cases: 25 Tried: 25 Errors: 0 Failures: 1 Skip:  0 Todo: 0 Timeouts: 0.

is there a type to store "ip in this network"?

when configuring and storing IP addresses, I would like to have a type to put the IP address and the subnet mask -- i.e. Ipaddr.V4.t * int -- but given the Prefix submodules, I was hoping to find a Ipaddr.V4.t * Ipaddr.V4.Prefix.t where the invariant that the ip address (first projection) is a member in the network (second projection), but I fail to find such a type. The Prefix.of_address_string constructs such a type (which does not carry that invariant along). Would be great to have this in a next release (unless it is already there).

ipaddr 2.7.1 opam install failed

Error messages as follow

The following actions will be performed:
  ∗  install ipaddr 2.7.1

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=  🐫
[ipaddr] Archive in cache

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=  🐫
[ERROR] The compilation of ipaddr failed at "ocaml pkg/pkg.ml build --pinned
        false --with-base-unix true".

#=== ERROR while installing ipaddr.2.7.1 ======================================#
# opam-version 1.2.2
# os           darwin
# command      ocaml pkg/pkg.ml build --pinned false --with-base-unix true
# path         /Users/liweijian/.opam/4.03.0/build/ipaddr.2.7.1
# compiler     4.03.0
# exit-code    1
# env-file     /Users/liweijian/.opam/4.03.0/build/ipaddr.2.7.1/ipaddr-11179-1872ec.env
# stdout-file  /Users/liweijian/.opam/4.03.0/build/ipaddr.2.7.1/ipaddr-11179-1872ec.out
# stderr-file  /Users/liweijian/.opam/4.03.0/build/ipaddr.2.7.1/ipaddr-11179-1872ec.err
### stdout ###
# [...]
# ocamlfind ocamlc -c -g -bin-annot -safe-string -principal -package 'bytes sexplib ppx_sexp_conv' -I lib -o lib/ipaddr_unix.cmo lib/ipaddr_unix.ml
# ocamlfind ocamlc -a -package 'bytes sexplib ppx_sexp_conv' lib/ipaddr_unix.cmo -o lib/ipaddr_unix.cma
# ocamlfind ocamldep -package 'bytes sexplib ppx_sexp_conv' -modules top/ipaddr_top.ml > top/ipaddr_top.ml.depends
# ocamlfind ocamldep -package 'bytes sexplib ppx_sexp_conv' -modules top/ipaddr_top.mli > top/ipaddr_top.mli.depends
# ocamlfind ocamlc -c -g -bin-annot -safe-string -principal -package 'bytes sexplib ppx_sexp_conv' -I top -I lib -o top/ipaddr_top.cmi top/ipaddr_top.mli
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -principal -package 'bytes sexplib ppx_sexp_conv' -I top -I lib -o top/ipaddr_top.cmx top/ipaddr_top.ml
# + ocamlfind ocamlopt -c -g -bin-annot -safe-string -principal -package 'bytes sexplib ppx_sexp_conv' -I top -I lib -o top/ipaddr_top.cmx top/ipaddr_top.ml
# File "top/ipaddr_top.ml", line 13, characters 16-45:
# Error: Unbound module Toploop
# Command exited with code 2.
### stderr ###
# [...]
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-tag' 'debug'
#      '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
#      'README.md' 'lib/ipaddr.a' 'lib/ipaddr.cmxs' 'lib/ipaddr.cmxa'
#      'lib/ipaddr.cma' 'lib/macaddr.cmx' 'lib/macaddr.cmi' 'lib/macaddr.mli'
#      'lib/ipaddr.cmx' 'lib/ipaddr.cmi' 'lib/ipaddr.mli' 'lib/ipaddr_unix.a'
#      'lib/ipaddr_unix.cmxs' 'lib/ipaddr_unix.cmxa' 'lib/ipaddr_unix.cma'
#      'lib/ipaddr_unix.cmx' 'lib/ipaddr_unix.cmi' 'lib/ipaddr_unix.mli'
#      'top/ipaddr_top.a' 'top/ipaddr_top.cmxs' 'top/ipaddr_top.cmxa'
#      'top/ipaddr_top.cma' 'top/ipaddr_top.cmx' 'top/ipaddr_top.cmi'
#      'top/ipaddr_top.mli']: exited with 10



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=  🐫
The following actions failed
  ∗  install ipaddr 2.7.1
No changes have been performed

classify routable and non-routable addresses

This distinguishes loopback and link-local in the IPv4 case. IPv6 needs investigation.

RFC 4193 Unique Local Address generation

We should probably be able to materialize an RFC 4193 ULA using the algorithm in the RFC.

See http://tools.ietf.org/html/rfc4193

index out of bounds with Ipaddr.V6.of_string

# Ipaddr.V6.of_string "[:";;
Exception: Invalid_argument "index out of bounds".

We should return an Error ... instead.

Inconsistency in V4.Prefix.of_string / V6.Prefix_of_string regarding trailing chars

This bit me today while working on a library to parse pf rules and was a little bit tricky to track down (with a trailing comma):

utop # Ipaddr.Prefix.of_string "172.16.0.0/12xxx";;
- : Ipaddr.Prefix.t option = Some 172.16.0.0/12

utop # Ipaddr.V4.Prefix.of_string "172.16.0.0/12xxx";;
- : Ipaddr.V4.Prefix.t option = None

EDIT: I forgot to mention that I prefer the latter (fail hard on invalid input). :-)

EDIT2: ping @djs55

Include cmti files.

Would be nice if ipaddr in opam installed the .cmti files so that merlin can pick up.

Ipaddr.to_domain_name returns an empty label

See: https://github.com/mirage/ocaml-ipaddr/blob/master/lib/ipaddr.ml#L227

I assume that the empty string was intended to signify an FQDN to distinguish it from a PQDN, but Dns.Name.marshal does not check for empty strings, leading it to encode two zero labels in the packet, which is invalid and cannot be parsed. I found this while writing a unit test for Mdns_resolver_mirage.gethostbyaddr, but Dns_resolver_mirage.gethostbyaddr has the same bug.

I'll raise a separate issue in ocaml-dns to discuss related points.

License file is missing

While the source indicates that this is MIT licensed, without the license file its hard to tell at a glance. For companies with a legal department it could also be an issue of which files are open and which are not. If you add a license file that just removes all the confusion.

Problem with int32-based comparison

For example, 150.65.76.42 is bigger than 150.65.76.43.
By using uint32 (from the uint package), we get these values: 2520861738 and 2520861739 where 2520861738 is actually smaller than 2520861739.

This is likely due to the fact that for IP v4 address values greater than 127.255.255.255, ordering based of Int32.compare is reversed compared to IP v4 address's order.

The easiest way to fix this would to use Uint32 but this add another dependency to the package and I am not sure if this is a good idea.

Here is a tentative Uint-based fix:
let compare a b =
Uint32.compare (Uint32.of_int32 (Ipaddr.V4.to_int32 a)) (Uint32.of_int32 (Ipaddr.V4.to_int32 b))

`Printexc.get_raw_backtrace` is >= 4.01.0 only

Workaround for 3.12 and 4.00.1.

explicit signatures of modules

Please provide explicit signatures of defined modules so that it is possible to extend the library.

Example: I have this Unbound value Ipaddr.V6.from_protobuf. I would like to be able to extend Ipaddr.V6 but currently there is no way to do it. I can extend Ipaddr, but not Ipaddr.V6