Comments (7)
Yes, unfortunately not yet possible. Meanwhile please use Claas' workaround.
We need to come up with a generic csv format perhaps including a header line(s) containing the device-ID. ActiPASS uses device-ID and then it can save and reuse device-calibration data.
See here to understand how ActiPASS parse ActiGraph files.
Anyone want to contribute with a generic CSV format definition and the corresponding ActiPASS parser? 😀
from actipass.
As far as I know, there is no possibility. Please correct me @pasan-h !
One way would be to reformat your acceleration data into an Actigraph .csv file format. I have done that in the past. You will need to specify the correct sampling frequency, start time and start date in the metadata header of the .csv file. I am happy to share some R code if you like.
from actipass.
@pasan-h I am more than happy to give it a go!
from actipass.
Here's my first draft for a generic CSV parser. The device ID and the sampling frequency are simply added as metadata in the first two lines of the CSV file. This can easily be done for a batch of files using R or Python. The sampling frequency could also be estimated from the timestamps otherwise. The acceleration axis would have to match the AX3 axis as specified by ActiPASS.
Even though the column names are not necessarily needed in the CSV file, I have added them for clarity.
I have also created a small test file: testdata.csv
function [Data, SF, deviceID] = readCSV(PATH)
% Read generic CSV files with specified format and ISO8601 timestamps
%
% Input:
% PATH [string]: full file path as a string
%
% Note: Input CSV file must have the shape of (n_rows, 4). Where
% n_rows is the number of sampled datapoints. The columns must be
% named [time, x, y, z] and arranged in the exact order.
%
% The time column must be in ISO8601 format (e.g. '2024-04-20T14:30.010Z')
%
% The acceleration axis must conform to the standard ActiPASS AX3
% orientation (i.e. x-axis pointing down).
%
% The first three lines in the CSV file must contain the
% deviceID and the sampling frequency and the column names:
% ID = <devideID>
% SF = <sampling frequency>
% time, x, y, z
%
%
% Output:
% Data [Nx4] datetime (Matlab datenum format) and 3D acceleration data
% SF [double] the sample frequency
% deviceID [string] the device ID
Data = [];
SF = NaN;
deviceID = NaN;
try
% Open the file once and keep it open for reading both metadata and data
fileID = fopen(PATH, 'r');
% Read the first two lines for metadata
idLine = fgetl(fileID);
sfLine = fgetl(fileID);
% Extract ID and SF values using correct indexing
deviceIDParts = strsplit(idLine, '= ');
deviceID = strtrim(deviceIDParts{2});
SFParts = strsplit(sfLine, '= ');
SF = str2double(strtrim(SFParts{2}));
% Skip the header line
fgetl(fileID);
% Use textscan to read the data, reading datetime as text
formatSpec = '%q%f%f%f';
dataArray = textscan(fileID, formatSpec, 'Delimiter', ',', 'EmptyValue', NaN);
fclose(fileID); % Ensure file is closed after reading
% Convert datetime text to datenum
timeText = dataArray{1};
timeFormat = 'yyyy-mm-ddTHH:MM:SS.FFFZ';
time = datenum(timeText, timeFormat);
% Prepare the output data
x = dataArray{2};
y = dataArray{3};
z = dataArray{4};
Data = [time, x, y, z];
catch APE
error("Error loading CSV file: " + APE.message);
end
from actipass.
Hi Claas,
Thanks very much for drafting the generic CSV import function. Now I have added the feature. I did some tests and realized the inclussion of a Time column (which is formatted as ISO8601 string) makes the CSV files about 80% big and also makes the loading considerably slower. Therefore I decided to ommit the time column. The consequence is that the format then only support resampled data to a fixed sample frequency. Since most accelerometer software supports resampling data to a fixed frequency before exporting to CSV anyway, I decided it's a good compromise. The source of the new function is here:
https://github.com/Ergo-Tools/ActiPASS/blob/main/utils/readGenericCSV.m
Cheers!
from actipass.
thanks Claas 🙏. I will try to add generic CSV import feature to to next ActiPASS release.
Things we need to think about:
- text parsing for big csv file would be slow (textscan). Perhaps we need to use a number format for time like CSVs from other devices. Perhaps the best is Unix epoch. Excel and matlab date numbers are also options. If we use Unix epochs, then we also need to define timezone of the measurement.
- The other options is if we can assume equi-distance samples (fixed sample-rate), then we do not need a time-column. But perhaps it's good to include a time column.
- I think some devices have missing data (like FirstBeat sensors with HR). Then perhaps we can add functionality to fill missing data to the parser. How about we use NaN to fill missing values?
from actipass.
Thanks, Pasan! I see it only as a first draft - especially since my MATLAB knowledge is fairly limited. From what I have read, textscan will be faster than e.g. readtable. But there will be a few other possibilities I am sure.
- I like the idea of switching the time to Unix time. That will be equally feasible. DateNum for example may be confusing for some users - since there is no straight forward way to convert timestamps to the DateNum format in R or Python as far as I know.
- Regarding the time: I'd argue to keep it in the file and use it for the subsequent resampling. Some users will assume their data has a fixed sample rate when infact it hasn't.
- The NaN option seems like a good idea. I guess we would have to make sure the missing data are all denoted the same in the CSV (i.e. empty value)
from actipass.
Related Issues (2)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from actipass.