juliamath/float8s.jl

A number format that you can count with your fingers.

Float8s.jl

Finally a number type that you can count with your fingers. Super Mario and Zelda would be proud.

Comes in two flavours: Float8 has 3 exponent bits and 4 fraction bits, Float8_4 has 4 exponent bits and 3 fraction bits. Both rely on conversion to Float32 to perform any arithmetic operation, similar to Float16.

CAUTION: Float8_4(::Float32) currently contains a bug for subnormals.

Example use

julia> using Float8s

julia> a = Float8(4)
Float8(4.0)

julia> b = Float8(3.14159)
Float8(3.125)

julia> a+b
Float8(7.0)

julia> sqrt(a)
Float8(2.0)

julia> a^2
Inf8

Most arithmetic operations are implemented. If you would like to have an additional feature, raise an issue.

Installation

Float8s.jl is registered, just do

pkg> add Float8s

Citation

This package was written for the following publication

Klöwer M, PD Düben and TN Palmer, 2020. Number formats, error mitigation and scope for 16-bit arithmetics in weather and climate modelling analyzed with a shallow water model, Journal of Advances in Modeling Earth Systems, 12, e2020MS002246. doi: 10.1029/2020MS002246

If you use this package in your own research, please cite us.

float8s.jl's Issues

promotion

would be nice if this worked:

julia> Float8(1.0) * Float32(1.0)
ERROR: promotion of types Float8 and Float32 failed to change any arguments

very useful package btw!

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Display bug?

With Julia 1.6-rc1:

julia> Float8(3.1)
Float8(0x49)

Float8_4 subnormal bug?

the README says there is one. is that still the case?

conversion

Just found this

julia> Float8(0.015)
Float8(0.0625)
julia> Float8(0.02)
Float8(0.015625)

although 0.015 <= 0.02 that's not true after conversion. Problem only occurs for subnormals. It's not the conversion back to Float32 (triggered by show):

 julia> bitstring(Float8(0.015))
"00000100"
julia> bitstring(Float8(0.02))
"00000001"

Need `eps(Float8)`

and eps(Float8_4)

5 bits for exponent?

the new H100 from nvidia has 8-bit floats in two flavors: 4 bits for the exponent like Float8s.jl's Float8_4, and 5 bits. scroll down to "NVIDIA Hopper FP8 data format" here: https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/

have you considered adding this type to Float8s.jl? currently i'm using https://github.com/goualard-f/MicroFloatingPoints.jl to simulate to see if that many exponent bits is better (than 4 or 3), and it is painfully slow.

Performance

This is currently only within x2 compared to Float16

julia> using BenchmarkTools, Float8s

julia> B1 = Float16.(rand(300,300));

julia> B2 = Float8.(rand(300,300));

julia> @btime Float32.($B1);
  281.829 μs (2 allocations: 351.64 KiB)

julia> @btime Float32.($B2);
  512.166 μs (2 allocations: 351.64 KiB)

julia> A = rand(Float32,300,300);

julia> @btime Float8.($A);
  981.933 μs (2 allocations: 88.02 KiB)

julia> @btime Float16.($A);
  673.638 μs (2 allocations: 175.89 KiB)

Maybe it helps to remove some of the if-clauses baked into function calls via multiple-dispatch, that are used to have type-flexibility for Float8 / Float8_4.

0xff NaN converted to 0

just found this

julia> Float8_4(0xff)
NaN8_4

julia> Float32(Float8_4(0xff))
0.0f0

julia> Float8(0xff)
NaN8

julia> Float32(Float8(0xff))
0.0f0

Which obv shouldn't happen.

register

@JuliaRegistrator register

Looking good?

Hey @JeffreySarnoff, following up on our conversation on slack, I adapted the Float16 code for Float8. As you have much more clue about floats than I have, would you mind having a look at it?