actigraph/pygt3x

Python module for reading GT3X/AGDC file format data

pygt3x's Issues

Support for GT3X v1

I received the following error when processing GT3X v1 https://github.com/actigraph/NHANES-GT3X-File-Format: couldn't find log.bin. Is GT3X V1 currently supported?

ValueError: 30 is not a valid Types

Stack trace:

    calibrated_reader = CalibratedReader(reader)
  File "<PYTHON_PATH>/site-packages/pygt3x/calibrated_reader.py", line 16, in __init__
    data = list(self.source.get_acceleration())
  File "<PYTHON_PATH>/site-packages/pygt3x/reader.py", line 72, in get_acceleration
    if Types(evt.header.eventType) == Types.Activity3:
  File "<PYTHON_PATH>/enum.py", line 315, in __call__
    return cls.__new__(cls, value)
  File "<PYTHON_PATH>/enum.py", line 611, in __new__
    raise ve_exc
ValueError: 30 is not a valid Types

I have 5 gt3x files throwing similar exception

assert self.idle_sleep_mode_activated

I am getting an assert error for some gt3x files:
reader.py -> line 164 -> assert self.idle_sleep_mode_activated

ValueError: cannot reshape array of size 5120 into shape (3)

Stack trace:

    calibrated_reader = CalibratedReader(reader)
  File "<PYTHON_PATH>/site-packages/pygt3x/calibrated_reader.py", line 16, in __init__
    data = list(self.source.get_acceleration())
  File "<PYTHON_PATH>/site-packages/pygt3x/reader.py", line 77, in get_acceleration
    payload = Activity1Payload(evt.payload, evt.header.timestamp)
  File "<PYTHON_PATH>/site-packages/pygt3x/activity_payload.py", line 11, in __init__
    self.AccelerationSamples = BitPackAcceleration.unpack_activity(
  File "<PYTHON_PATH>/site-packages/pygt3x/bit_pack_acceleration.py", line 27, in unpack_activity
    data = concat.reshape((-1, 3))
ValueError: cannot reshape array of size 5120 into shape (3)

I have in-total 6 gt3x files throwing similar exceptions.

ValueError: cannot reshape array of size 1 into shape (0,3) ==> at version = "0.1.6", however no error at version = "0.1.1"

Hi,

If I use the latest version (version = "0.1.6") I obtain:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [3], in <cell line: 3>()
      1 col_names = ["Timestamp", "X", "Y", "Z"]
      3 with FileReader(file) as reader:
----> 4     calibrated_reader =  CalibratedReader(reader)
      5     data = calibrated_reader.get_samples()
      7     df_l = pd.DataFrame(data, columns=col_names)

File ~/pygt3x/calibrated_reader.py:16, in CalibratedReader.__init__(self, source)
     14 def __init__(self, source: FileReader):
     15     self.source = source
---> 16     data = list(self.source.get_acceleration())
     17     if len(data) == 0:
     18         self.acceleration = np.empty((0, 4))

File ~/pygt3x/reader.py:78, in FileReader.get_acceleration(self, num_rows)
     76     payload = Activity2Payload(evt.payload, evt.header.timestamp)
     77 elif Types(evt.header.eventType) == Types.Activity:
---> 78     payload = Activity1Payload(evt.payload, evt.header.timestamp)
     79 else:
     80     continue

File ~/pygt3x/activity_payload.py:11, in Activity1Payload.__init__(self, payload_bytes, timestamp)
     10 def __init__(self, payload_bytes, timestamp):
---> 11     self.AccelerationSamples = BitPackAcceleration.unpack_activity(
     12         payload_bytes, timestamp, True
     13     )

File ~/pygt3x/bit_pack_acceleration.py:22, in BitPackAcceleration.unpack_activity(source, timestamp, flip_xy)
      8 """
      9 Unpacks activity stored as sets of 3, 12-bit integers
     10 
   (...)
     18 
     19 """
     20 data = np.frombuffer(source, dtype=np.uint8)
     21 fst_uint8, mid_uint8, lst_uint8 = (
---> 22     np.reshape(data, (data.shape[0] // 3, 3)).astype(np.int16).T
     23 )
     24 fst_uint12 = (fst_uint8 << 4) + (mid_uint8 >> 4)
     25 snd_uint12 = ((mid_uint8 % 16) << 8) + lst_uint8

File <__array_function__ internals>:180, in reshape(*args, **kwargs)

File ~/miniconda3/envs/pygt3x_venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py:298, in reshape(a, newshape, order)
    198 @array_function_dispatch(_reshape_dispatcher)
    199 def reshape(a, newshape, order='C'):
    200     """
    201     Gives a new shape to an array without changing its data.
    202 
   (...)
    296            [5, 6]])
    297     """
--> 298     return _wrapfunc(a, 'reshape', newshape, order=order)

File ~/miniconda3/envs/pygt3x_venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py:57, in _wrapfunc(obj, method, *args, **kwds)
     54     return _wrapit(obj, method, *args, **kwds)
     56 try:
---> 57     return bound(*args, **kwds)
     58 except TypeError:
     59     # A TypeError occurs if the object does have such a method in its
     60     # class, but its signature is not identical to that of NumPy's. This
   (...)
     64     # Call _wrapit from within the except clause to ensure a potential
     65     # exception has a traceback chain.
     66     return _wrapit(obj, method, *args, **kwds)

ValueError: cannot reshape array of size 1 into shape (0,3)

However, if I use version = "0.1.1" I can read the files.

My conda env:

name: pygt3x_venv
channels: 
  - conda-forge
  - defaults
dependencies:
  - python=3.8
  - pip=21.2.4
  - pandas=1.0.5
  - jupyterlab=3.2.6
  - matplotlib=3.5.1

reader.read_info() is skipping TimeZone

The format of TimeZone inside gt3x file is this: "TimeZone: -04:00:00"

read_info() function is skipping "TimeZone" because of the following check:

values = line.split(":")
if len(values) == 2: