bernardmetzler/geds

GEDS is a distributed ephemeral data store that enables flexible scaling of compute and storage. It uses a centralized name-node to store active objects and transparently integrates into S3-like object-stores.

GEDS - A Generic Ephemeral Data Store

GEDS is a distributed epehemeral datastore that enables flexible scaling of compute and storage. It uses a centralized namenode to store active objects and transparently integrates into S3-like object-stores.

• Building
• geds_smart_open Python package

Ports

GEDS uses the following default ports (see also src/utility/Ports.h).

• Web / Prometheus: 4380
• Metadata Server: 4381
• GEDS RPC: 4382

Benchmarking

1. Start the metadata server on $HOST:

   $GEDS_INSTALL/bin/metadataserver
   

2. Start the benchmarking server on a separate node

   GEDS_METADATASERVER_HOST=$HOST
   GEDS_METADATASERVER_PORT=4381
   export GEDS_METADATASERVER=${GEDS_METADATASERVER_HOST}:${GEDS_METADATASERVER_PORT}
   cd $GEDS_INSTALL/python
   python3.9 serve_benchmark.py
   

3. Run the benchmark command:

   GEDS_METADATASERVER_HOST=$HOST
   GEDS_METADATASERVER_PORT=4381
   $GEDS_INSTALL/bin/benchmark_io --address $GEDS_METADATASERVER_HOST --outputFile output.csv

Contributing

Contributing to this repository requires a signed contributor license agreement (see GEDS_CLA_Corporate for corporate copyright holders or GEDS_CLA_Individual).

All commits need to contain signed-off message to indicate that the submitter accepts the DCO. Example

Commit message

Signed-off-by: John Doe <[email protected]>

The command git commit -s will automatically add the message to the end of the commit.

This project welcomes external contributions. To contribute code or documentation, please submit a pull request.

Acknowledgement

This project has received funding through CloudStars.

CloudStars has received funding from the European Union's Horizon research and innovation programme under grant agreement No 101086248.