janssenhenning/aiida-dataframe

AiiDA data plugin for pandas DataFrame objects

aiida-dataframe

AiiDA data plugin for pandas DataFrame objects

Features

• Store pandas.DataFrame objects in the Database:

  import pandas as pd
  PandasFrameData = DataFactory('dataframe.frame')
  df = pd.DataFrame(
       {
           "A": 1.0,
           "B": pd.Timestamp("20130102"),
           "C": pd.Series(1, index=list(range(4)), dtype="float32"),
           "D": np.array([3] * 4, dtype="int32"),
           "E": pd.Categorical(["test", "train", "test", "train"]),
           "F": "foo",
       }
   )
  df_node = PandasFrameData(df)
  df_node.store()

• Retrieving the pandas.DataFrame from the Database :

  from aiida.orm import QueryBuilder
  df_node = QueryBuilder().append(PandasFrameData).first()[0]
  df = df_node.df #The df property reconstructs the pandas DataFrame
  print(df.head())

Installation

pip install aiida-dataframe
verdi quicksetup  # better to set up a new profile
verdi plugin list aiida.data  # should now show your data plugins

Usage

The plugin also includes verdi commands to inspect its data types:

verdi data dataframe list
verdi data dataframe export <PK>
verdi data dataframe show <PK>

Development

git clone https://github.com/janssenhenning/aiida-dataframe .
cd aiida-dataframe
pip install --upgrade pip
pip install -e .[pre-commit,testing]  # install extra dependencies
pre-commit install  # install pre-commit hooks
pytest -v  # discover and run all tests

See the developer guide for more information.

License

MIT

Contact

[email protected]

aiida-dataframe's Issues

df only columns

I would like to use aiida-dataframe in my porting of crystal structure search package, CrySPY,
but I have a trouble. Is it possible to change your code to handle dataframe without any data?
An example,

import pandas as pd
df = pd.DataFrame([], columns=["x","y"])
df
PandasFrameData = DataFactory('dataframe.frame')
framedata = PandasFrameData(df) 

causes
AttributeError: 'PandasFrameData' object has no attribute '_df'

An error when using restartworkchain

Maybe it is because I use still aiida __version__='1.6.9', but an error occurs after all the calculations are successfully finished when using restartworkchain.

A sample script is in sample/restart.py in
https://github.com/nim-hrkn/aiida-dataframe.

The error occurs when

_OUTPUT_DF = True

which outputs FrameData, but no error when

_OUTPUT_DF = False

which doesn't output FrameData.

The Error messages are

...
  File "/usr/lib/python3.7/asyncio/tasks.py", line 249, in __step
    result = coro.send(None)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/processes.py", line 1230, in step_until_terminated
    await self.step()
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/processes.py", line 1216, in step
    self.transition_to(next_state)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/base/state_machine.py", line 335, in transition_to
    self.transition_failed(initial_state_label, label, *sys.exc_info()[1:])
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/base/state_machine.py", line 351, in transition_failed
    raise exception.with_traceback(trace)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/base/state_machine.py", line 320, in transition_to
    self._enter_next_state(new_state)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/base/state_machine.py", line 386, in _enter_next_state
    self._fire_state_event(StateEventHook.ENTERED_STATE, last_state)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/base/state_machine.py", line 299, in _fire_state_event
    callback(self, hook, state)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/plumpy/processes.py", line 326, in <lambda>
    lambda _s, _h, from_state: self.on_entered(cast(Optional[process_states.State], from_state)),
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/aiida/engine/processes/process.py", line 387, in on_entered
    self.update_node_state(self._state)
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/aiida/engine/processes/process.py", line 610, in update_node_state
    self.update_outputs()
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/aiida/engine/processes/process.py", line 635, in update_outputs
    output.store()
  File "/home/max/.virtualenvs/py39/lib/python3.7/site-packages/aiida_dataframe/data/dataframe.py", line 134, in store
    current_hash = self._hash_dataframe(self._df)
AttributeError: 'PandasFrameData' object has no attribute '_df'

I use aiida-lammps for development and still use AiiDA v1.x.