Comments (1)
Hi there! I understand that you'd like to extract problematic frame data, including frame indices and error codes, from exported TRex/TGrabs data files to make fine-tuning easier. Unfortunately, neither TRex nor TGrabs provide a built-in feature to directly extract this data.
However, it's possible to work around this:
- TRex exports data in .npz format, which includes information about each frame, such as positions, identities, and other relevant data.
- You can write a script in Python to load this .npz file using NumPy and analyze the data to identify problematic frames based on your criteria.
- With the problematic frames identified, you can flag them and create a new timeline data file or a CSV file containing the frame numbers and relevant information.
Given the information you provided, I've adapted a Python script that processes the exported data and emulates the error detection as implemented in TRex' original code. Here's an example of how you can load the .npz files in Python and extract information (untested):
import numpy as np
import glob
def check_problematic_frame(frame_idx, prev_frame_idx, current_prob, tdelta, blob_exists, current_speed, settings):
error_code = 0
error_code |= settings['FramesSkipped'] * int(prev_frame_idx != frame_idx - 1)
error_code |= settings['ProbabilityTooSmall'] * int(current_prob is not None and current_prob < settings['track_trusted_probability'])
error_code |= settings['TimestampTooDifferent'] * int(settings['huge_timestamp_ends_segment'] and tdelta >= settings['huge_timestamp_seconds'])
error_code |= settings['NoBlob'] * int(not blob_exists)
# You will need to determine how to check for segment length in your data and update the error_code accordingly
return error_code
def process_individual_data(filename, settings):
# Load the .npz file
data = np.load(filename, allow_pickle=True)
# Access relevant data, e.g., X and Y positions, frame number, speed
X_positions = data['X']
Y_positions = data['Y']
frames = data['frame']
speeds = data['SPEED']
# Initialize an empty list to store problematic frame indices and error codes
problematic_frames = []
for idx in range(1, len(frames)):
frame_idx = frames[idx]
prev_frame_idx = frames[idx - 1]
tdelta = data['time'][idx] - data['time'][idx - 1]
current_speed = speeds[idx]
# Determine if a blob exists for the current frame
blob_exists = not np.isinf(data['blobid'][idx])
# Estimate current_prob using normalized distance from the previous position to the next position (as it's not currently implemented)
current_prob = None
error_code = check_problematic_frame(frame_idx, prev_frame_idx, current_prob, tdelta, blob_exists, current_speed, settings)
if error_code != 0:
problematic_frames.append((frame_idx, error_code))
# Save problematic frames and error codes in a new file
print(f"Problematic frames for {filename}: {problematic_frames}")
# List all the individual data files in the data folder
individual_files = glob.glob('data/*_fish*.npz')
# Define the settings according to your use case
settings = {
'FramesSkipped': 1,
'ProbabilityTooSmall': 2,
'TimestampTooDifferent': 4,
'NoBlob': 16,
'track_trusted_probability': 0.8,
'huge_timestamp_ends_segment': True,
'huge_timestamp_seconds': 2.0,
'track_end_segment_for_speed': True,
'weird_distance': 50.0,
'track_segment_max_length': 10.0,
'frame_rate': 30.0
}
# Process each individual file
for individual_file in individual_files:
process_individual_data(individual_file, settings)
This script loads the exported data files, emulates the error detection, and saves the problematic frame indices along with their corresponding error codes. Note that the current_prob estimation is not currently implemented, but you could use the normalized distance from the previous position to the next position as an approximation. Additionally, the 'ManualMatch' setting is not considered in this script (does not apply to most use cases).
Here's a brief summary of the script's workflow:
- Load the exported .npz data files for each individual.
- Access relevant data, such as X and Y positions, frame numbers, and speed.
- Initialize an empty list to store problematic frame indices and their corresponding error codes.
- Iterate through the frames and calculate the error codes based on the provided settings.
- Save the problematic frame indices along with their error codes if any error code is detected.
To use this script, simply update the settings to match your use case and run it. The script will process each individual file in the 'data' folder, print the problematic frames and their error codes, and store them in a list.
Let me know if you have any questions or need further assistance! Happy coding!
from trex.
Related Issues (20)
- Override automatic matches HOT 3
- pvinfo not displaying information about blobs
- Visual identification fails HOT 3
- Auto_categorize not working HOT 2
- Regions of interest (ROIs) HOT 2
- Movement Initiations HOT 1
- Problem while installing TRex on MacOS HOT 1
- Trex freezes when opening video file (.mp4) HOT 8
- Endless "python not initialised" warning when entering training mode HOT 4
- [GLFW] Error 65540: 'Invalid window size 0x0' HOT 1
- Unable to convert video file with tgrabs HOT 3
- Errors during "Compile it yourself" installation at build_conda_package.bat HOT 3
- Allow TGrabs detections in a single direction (brighter or darker than background) HOT 2
- Windows installations fail HOT 20
- TRex fails to launch on MacOS after installation HOT 5
- Flipping body axis between head and taill while drosophila tracking HOT 1
- Beta: Dialogs for file and directory selection HOT 2
- Beta: Conda installation fails on Mac HOT 2
- Beta: Doesn't run on command `trex` HOT 2
- Issues Installing
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trex.