klermonte/zerg

PHP structured binary parser

zerg

Zerg is a small PHP tool that allow you simply parse structured binary files like lsdj memory dump file, jpeg encoded image or your custom binary format file.

Introdution

If you are reading this, chances are you know exactly why you need to read binary files in PHP. So I will not explain to you that this is not a good idea. Nevertheless, I like you needed to do this is in PHP. That's why I create zerg project. During creation, I was inspired by following projects: alexras/bread and themainframe/php-binary.

Installation

composer require klermonte/zerg dev-master
Or add "klermonte/zerg": "dev-master" to your dependancy list in composer.json and run composer update

Usage

// Describe your binary format in zerg language
$fieldCollection = new \Zerg\Field\Collection([
    'stringValue' => ['string', 15],
    'intValue' => ['arr', 5, ['int', 8]],
    'enumValue' => ['enum', 8, [
            0 => 'zero',
            10 => 'ten',
            32 => 'many'
        ], ['default' => 'not found']
    ]
]);

// Wrap your data in one of zerg streams
$sourceStream = new \Zerg\Stream\StringStream("Hello from zerg123456");

//Get your data structure
$data  = $fieldCollection->parse($sourceStream);
print_r($data);
/*
Array
(
    [stringValue] => Hello from zerg
    [intValue] => Array
        (
            [0] => 49
            [1] => 50
            [2] => 51
            [3] => 52
            [4] => 53
        )

    [enumValue] => not found
)
*/

Field types

Integer

// Object notation
// --------------------------------------
// $field = new Int(<size>, <options>);

$field = new Int(4);
$field = new Int('byte', [
    'signed' => true, 
    'formatter' => function($value) {
        return $value * 100;
    }
]);

// Array notation
// --------------------------------------
// $fieldArray = ['int', <size>, <options>];

Avaliable options

Option name	Avaliable values	Description
signed	boolean, default false	Whether field value is signed or not
endian	PhpBio\Endian::ENDIAN_BIG or PhpBio\Endian::ENDIAN_LITTLE	Endianess of field
formatter	callable	callback, that take 2 arguments: function ($parsedValue, $dataSetInstance) {...}

String

// Object notation
// --------------------------------------
// $field = new String(<size>, <options>);

$field = new String(16);
$field = new String('short', [
    'endian' => PhpBio\Endian::ENDIAN_BIG, 
    'formatter' => function($value) {
        return str_repeat($value, 2);
    }
]);

// Array notation
// --------------------------------------
// $fieldArray = ['string', <size>, <options>];

Avaliable options

Option name	Avaliable values	Description
endian	PhpBio\Endian::ENDIAN_BIG or PhpBio\Endian::ENDIAN_LITTLE	Endianess of field
formatter	callable	callback, that take 2 arguments: function ($parsedValue, DataSet $dataSet) {...}

Padding

// Object notation
// --------------------------------------
// $field = new Padding(<size>);

$field = new Padding(16);

// Array notation
// --------------------------------------
// $fieldArray = ['padding', <size>];

Enum

// Object notation
// --------------------------------------
// $field = new Enum(<size>, <values>, <options>);

$field = new Enum(8, [0, 1, 2, 3]);
$field = new Enum('short', [
        1234 => 'qwerty1',
        2345 => 'qwerty2'
    ], [
        'default' => 'abcdef'
    ]
);

// Array notation
// --------------------------------------
// $fieldArray = ['enum', <values>, <options>];

Avaliable options

Option name	Avaliable values	Description
default	mixed, optional	Value, that will be returned, if no one key from values matchs to parsed value

And all options from Integer field type.

Conditional

// Object notation
// --------------------------------------
// $field = new Conditional(<key>, <fields>, <options>);

$field = new Conditional('/path/to/key/value', [
        1 => ['int', 32],
        2 => ['string', 32]
    ], [
        'default' => ['padding', 32]
    ]
);

// Array notation
// --------------------------------------
// $fieldArray = ['conditional', <fields>, <options>];

Avaliable options

Option name	Avaliable values	Description
default	array, optional	Field in array notation, that will be used, if no one key from field matchs to parsed value

Array

// Object notation
// --------------------------------------
// $field = new Arr(<count>, <field>, <options>);

$field = new Arr(10, ['int', 32]);

// Array notation
// --------------------------------------
// $fieldArray = ['arr', <field>, <options>];

Avaliable options

Option name	Avaliable values	Description
until	'eof' or callable	If set, array field count parameter will be ignored, and field will parse values until End of File or callback return false, callback take one argument: function ($lastParsedValue) {...}

Collection

// Object notation
// --------------------------------------
// $field = new Collection(<fields>, <options>);

$field = new Collection([
    'firstValue' => ['int', 32],
    'secondValue' => ['string', 32]
]);

// Array notation
// --------------------------------------
// $fieldArray = ['collection', <fields>, <options>];
// or just
// $fieldArray = <fields>;

Back links

Size, count and conditional key parameters may be declared as a back link - path to already parsed value. Path can starts with / sign, that means root of data set or with '../' for relative path.

$fieldCollection = new \Zerg\Field\Collection([
    'count' => ['string', 2],
    'intValue' => ['arr', '/count', ['int', 8]]
]);
$sourceStream = new \Zerg\Stream\StringStream("101234567890");
$data = $fieldCollection->parse($sourceStream);
print_r($data);
/*
Array
(
    [count] => 10
    [intValue] => Array
        (
            [0] => 49
            [1] => 50
            [2] => 51
            [3] => 52
            [4] => 53
            [5] => 54
            [6] => 55
            [7] => 56
            [8] => 57
            [9] => 48
        )
)
*/

Conditional example

$fieldCollection = new \Zerg\Field\Collection([
    'count' => ['string', 2],
    'conditional' => ['conditional', '/count', [
            0 => ['string', 80],
            10 => ['int', 16]
        ],
        [
            'default' => ['string', 2]
        ]
    ]
]);
$sourceStream = new \Zerg\Stream\StringStream("101234567890");
$data = $fieldCollection->parse($sourceStream);
print_r($data);
/*
Array
(
    [count] => 10
    [conditional] => 12849
)
*/

zerg's Issues

Size and count getters merge

Merge similar methods getSize and getCount to one

Add resolving of relative path

Currently \Zerg\DataSet process only absolute path, that starts from / and search value form root.
dataset should resolve path like ./../a/b/c../../d/e/f or just ./a thats mean sibling name of a current position of dataset.

Add validation functionality

Remove duplication in code

https://scrutinizer-ci.com/g/klermonte/zerg/issues/master?selectedLabels%5B0%5D=4

Refactor src/Zerg/Field/CountableTrait.php from a C on Code Climate

https://codeclimate.com/github/klermonte/zerg/src/Zerg/Field/CountableTrait.php

Not cache resolved property by relative path

Optimize autoloads

maybe use class map

explicitly pass DataSet to parse method insted implicitly DataSet setting

Saving data to existing or new stream

Prevent dublicate instance chosen field in conditional field

Field repeating based on validation

Refactoring

• rename AbstractField::setMainParam() to AbstractField::init()
• change visibility of Scalar::format() and Scalar::parseSizeWord() to private
• cast processed value to int in AbstractField::getSize() and AbstractField::getCount()
• remove unused use statements

Implement 64bit int

Fully document all classes, methods and properties

Add float field

Test ticket from Code Climate

New path format

Insted file path style use object notation:
/header/blocksCount => @root.header.blocksCount or just header.blocksCount
./blocksCount => @parent.blocksCount
suggest new types of path parts:

• @parent - select parent of current field
• @prev - select previous sibling
• @root - select root of DataSet, optional, just for clarity e.g. @root.header and header are equal

Mask field

Field to extract bitwise parts from one (or up to 4 or 8 depends php capacity) byte.
Field declaration:

$field = ['mask', $size, [
    'fields' => [
        $lowSkipBits,
        'low' => $lowBits,
        $skipBits,
        'hight' => $hightBits
    ]
]]; 

Features:

• $size in BYTES (just like in string field)
• $size MUST be no more than maximum machine (php interpretor) byte capacity (4 for 32 bits or 8 for 64 bits)
• sub field size in bits
• fields with numeric key are skipped (just like padding field)
• sum of fields sizes may be less than $size

Return value: assoc array of bit fields values

Write README

Nested array repeat

[
    'count' => [255, 16] // means 255 arrays of 16 elements in each
    // we can set as many nest levels as we need
    'count' => [1, 2, 10, 5]
    // sure we can use back links
    'count' => ['/header/elementsCount', 16]
    // or we can use validation based repeat when it will be implemented
    'count' => [$validataionStuff, 128, 8]
]

Extract field property aunctionality to separate class