Comments (1)
Hi there! I understand that you'd like to extract problematic frame data, including frame indices and error codes, from exported TRex/TGrabs data files to make fine-tuning easier. Unfortunately, neither TRex nor TGrabs provide a built-in feature to directly extract this data.
However, it's possible to work around this:
- TRex exports data in .npz format, which includes information about each frame, such as positions, identities, and other relevant data.
- You can write a script in Python to load this .npz file using NumPy and analyze the data to identify problematic frames based on your criteria.
- With the problematic frames identified, you can flag them and create a new timeline data file or a CSV file containing the frame numbers and relevant information.
Given the information you provided, I've adapted a Python script that processes the exported data and emulates the error detection as implemented in TRex' original code. Here's an example of how you can load the .npz files in Python and extract information (untested):
import numpy as np
import glob
def check_problematic_frame(frame_idx, prev_frame_idx, current_prob, tdelta, blob_exists, current_speed, settings):
error_code = 0
error_code |= settings['FramesSkipped'] * int(prev_frame_idx != frame_idx - 1)
error_code |= settings['ProbabilityTooSmall'] * int(current_prob is not None and current_prob < settings['track_trusted_probability'])
error_code |= settings['TimestampTooDifferent'] * int(settings['huge_timestamp_ends_segment'] and tdelta >= settings['huge_timestamp_seconds'])
error_code |= settings['NoBlob'] * int(not blob_exists)
# You will need to determine how to check for segment length in your data and update the error_code accordingly
return error_code
def process_individual_data(filename, settings):
# Load the .npz file
data = np.load(filename, allow_pickle=True)
# Access relevant data, e.g., X and Y positions, frame number, speed
X_positions = data['X']
Y_positions = data['Y']
frames = data['frame']
speeds = data['SPEED']
# Initialize an empty list to store problematic frame indices and error codes
problematic_frames = []
for idx in range(1, len(frames)):
frame_idx = frames[idx]
prev_frame_idx = frames[idx - 1]
tdelta = data['time'][idx] - data['time'][idx - 1]
current_speed = speeds[idx]
# Determine if a blob exists for the current frame
blob_exists = not np.isinf(data['blobid'][idx])
# Estimate current_prob using normalized distance from the previous position to the next position (as it's not currently implemented)
current_prob = None
error_code = check_problematic_frame(frame_idx, prev_frame_idx, current_prob, tdelta, blob_exists, current_speed, settings)
if error_code != 0:
problematic_frames.append((frame_idx, error_code))
# Save problematic frames and error codes in a new file
print(f"Problematic frames for {filename}: {problematic_frames}")
# List all the individual data files in the data folder
individual_files = glob.glob('data/*_fish*.npz')
# Define the settings according to your use case
settings = {
'FramesSkipped': 1,
'ProbabilityTooSmall': 2,
'TimestampTooDifferent': 4,
'NoBlob': 16,
'track_trusted_probability': 0.8,
'huge_timestamp_ends_segment': True,
'huge_timestamp_seconds': 2.0,
'track_end_segment_for_speed': True,
'weird_distance': 50.0,
'track_segment_max_length': 10.0,
'frame_rate': 30.0
}
# Process each individual file
for individual_file in individual_files:
process_individual_data(individual_file, settings)
This script loads the exported data files, emulates the error detection, and saves the problematic frame indices along with their corresponding error codes. Note that the current_prob estimation is not currently implemented, but you could use the normalized distance from the previous position to the next position as an approximation. Additionally, the 'ManualMatch' setting is not considered in this script (does not apply to most use cases).
Here's a brief summary of the script's workflow:
- Load the exported .npz data files for each individual.
- Access relevant data, such as X and Y positions, frame numbers, and speed.
- Initialize an empty list to store problematic frame indices and their corresponding error codes.
- Iterate through the frames and calculate the error codes based on the provided settings.
- Save the problematic frame indices along with their error codes if any error code is detected.
To use this script, simply update the settings to match your use case and run it. The script will process each individual file in the 'data' folder, print the problematic frames and their error codes, and store them in a list.
Let me know if you have any questions or need further assistance! Happy coding!
from trex.
Related Issues (20)
- Trex Quitting Unexpectedly + TGrabs not showing up on Trex HOT 3
- Export fish path HOT 1
- How is neighbor distance calculated? HOT 1
- Proximity Issue HOT 1
- libGL error on fedora 36 HOT 3
- Error in Local conda channel Installation HOT 1
- Heatmap not working after update to 1.1.9 HOT 2
- manually setting origin HOT 4
- frame_rate doesn't seem to map predictably across parameters HOT 2
- Additional info on blob values in raw mode HOT 3
- Size of GUI HOT 3
- File format for visual fields HOT 1
- Training images not being saved HOT 2
- TRex crash HOT 3
- Override automatic matches HOT 3
- pvinfo not displaying information about blobs
- Visual identification fails HOT 3
- Auto_categorize not working HOT 2
- Open file dialog upon Load Settings HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trex.