Switch to a faster result data serialization format

[tum-ens]

Switch to a faster result data serialization format

[ojdo]

Inventory

Kind of obvious (in hindsight): simply call get_entity once for each model entity returned by a call to list_entities for each of the entity types set, parameter, variable, objective, constraint. Using this, no manual maintenance is required and all possibly interesting information is preserved.

Serialization format

Pandas supports HDF5 by relying on the PyTables package.

Internal result representation

A (possibly nested) dictionary of pandas Series:

result = {
    'set': { 't': <Series>, 'tm': <Series>, 'com': <Series>, ... },
    'par': { 'w': <Series>, ... },
    'var': { ... },
    'obj': { ... },
    'con': { ... }}

Alternative: a nested AttrDict for easier access like result.var.e_pro_out.

Further thoughts

The choice of representation has ripple effects concerning functions get_timeseries and get_constants, which should switch to relying on the result representation that gets automatically updated by a solver run. This in turn calls for creating a custom class (urbs, maybe?) that bundles this new behaviour. Then, an urbs object could either be created in its solved state by reading input + solving, or by directly loading a HDFStore file, populating the internal result data structure. Additional meta information (time stamps, scenario descriptions, notes) could transparently be added.

[ojdo]

Update on internal result representation: the dictionary probably should not be nested by entity type, to make the logic of function get_entity simpler, like:

def get_entity(instance, name):
    if name in instance._result:
        return instance._result[name]
    else:
        raise SomeException

The hidden _result field should get populated automatically after a successful solver run. Perhaps, the output of list_entities (entity name, type and domain) could be stored where?