calebsander/structure-bytes

A TypeScript library for more efficient data transfers by separating the structure from the values and storing each as binary data

structure-bytes's Issues

Add README

Simpler enum of strings syntax

Rename lengthBuffer

The local variable is named the same as the function in structure-types.js

Add enum wrappers around each type

Split off jQuery

Add an .npmignore to remove test files, etc.

Let OptionalType value be undefined

Turn assert.assert() into assert()

Can probably assert that reading value consumes whole buffer

May need a special case for the CharType test

Add test for hasOwnProperty in StructType construction

Cap `TupleType` length at 256

This would save space since it is unlikely that a specific large number of instances of a type would be needed. If longer tuples are necessary, you can always transform them into tuples of tuples.

Add examples

Can just do require('./module') for relative requires

Add readTypeAndValue

Would be used by client-side on download

Add istanbul ignore for Simultaneity error lines

Fix undefined not working as field value in struct with optional field

This is because we are directly comparing field values to undefined for better error messages. Suggested solution: If an error is thrown, then check whether the field value was defined. If it was undefined, throw the "field doesn't exist" error; otherwise, throw whatever the original error was.

Add comments

Add compilation to npm test

Implement ChoiceType where each type is associated with a class

This would work like enums in Swift. If UPC and QRCode are constructors:

new sb.ChoiceType(new Map()
  .set(UPC, new sb.StructType({
    code: new sb.TupleType({type: new sb.IntType, length: 4})
  }))
  .set(QRCode, new sb.StructType({
    code: new sb.StringType
  }))
)

If the old syntax is used, each type would get assigned a dummy constructor name. The names of each of the constructors would be serialized alongside the corresponding types. Have to figure out how reading the value would indicate what type it was originally.

Remove client-side reliance on util.inspect()

Allow numeric types to take in valid strings

Add length validations to consumeValue

Add util.inspect for Set, Map

Add DayType for just storing date and TimeType for just storing time

TimeType would store a 4-byte integer; DayType would store a 3-byte integer

Is it possible to do recursive types?

It is currently impossible to instantiate a recursive type because types are immutable, so you would have to have the type already created in order to pass it into its own construction.

Idea:

• Have a global registry of named types
• To use a type in its own definition, would use new sb.RecursiveType(typeName)
• Then add the type to the global registry

Example:

sb.registerType({
  type: new sb.SetType(
    new sb.RecursiveType('node')
  ),
  name: 'node'
})

When the type is written, the name (or equivalent identifier) probably has to be written with the first instance, and then when it is repeated, it would refer to that identifier to specify what type to use.

Each value write buffer would need to keep track of the positions in which recursive types' values were written. If the value was already written, a pointer would be created to it. Otherwise, the value would have to be added to the set of pointers to fill in later.

Disallow Boolean*Type being used as a RecursiveType because there is no recursion possible

Make throws assertions say something about the error message

Make readValue a method of Type

Add tests for GrowableBuffer and BufferStream

Add equality functions for types

Use unsigned numbers for Date and Day

Readd coverage in read.js

Test reading value for new sb.BooleanArrayType where the value buffer is [0, 0, 1]

Fix indentation on test files

Add download support

General idea:

• Cache type and signature in localStorage
• In request, add header containing signature
• Response header should contain current type signature (even if unchanged)
• If signature has changed, use writeTypeAndValue(). Otherwise, use writeValue().

Possible improvements to spec

• Octets alias for Array<UnsignedByte>
• Store strings as null-terminated entities
• Add arbitrary-precision int and unsigned int
• 0xFF type byte would signal that the next 2 bytes are an unsigned short containing the offset to where the type can be found. This would allow for smaller type declarations if the same types are used in several places.
• Date type (64-bit unsigned integer, in ms)
• Union type

Purge parentheses after new?

Add ExtendedType for making a subclass of a StructType

Add note about testing to README

npm test and node client-test/server.js

Object wrappers around arguments

e.g.

new t.TupleType({type: new t.ByteType(), length: 20})

Could the StructType constructor be made nicer?

Since the fields have distinct names, consider using:

new t.StructType({
  a: new t.IntType(),
  b: new t.StringType()
})

and then just convert to an array and sort by field name.

Look into using Closure Compiler instead of Uglify

Update README with ArrayBuffer

Use typed arrays rather than buffers to make browser code smaller

Will require a lot of changes across the project, upload.js will need to be modified

Add documentation of binary format in README with some examples

Cache pointer reads

Only gzip HTTP response if client allows the encoding

Add documentation for client functions

More efficient int representation

0b0XXXXXXX -> 1 byte integer (0 to 2^7 - 1)
0b10XXXXXX XXXXXXXX -> 2 byte integer (2^7 to 2^7 + 2^14 - 1)
Etc. up to 8 byte integers
Use this for lengths, offsets, etc. as well

Add headers and explanation of versioning to README

Ensure synthetic recursive type names don't conflict with existing ones