Comments (7)
Hi, @bahanonu, thanks for pointing out the problem. there is an option for avoiding saving intermediate results.
you can set neuron.options.save_intermediate = false. I think by default this value is false.
Pengcheng
from cnmf_e.
Thanks. However, even with save_intermediate
false (as it is by default https://github.com/zhoupc/CNMF_E/blob/master/ca_source_extraction/CNMFSetParms.m#L308), an intermediate folder is still created with the mat-file copy of the movie data and is not removed at the end. What does save_intermediate
actually do?
Please re-open the issue until this is resolved.
from cnmf_e.
Or do you have to run neuron.getReady_batch
instead of neuron.getReady
to prevent creation of an intermediate folder? e.g. will neuron.getReady
always create a mat-file regardless of the save_intermediate
flag?
from cnmf_e.
there are following folders/files created for running CNMF-E:
-
data_x_x_x.mat
this file is a copy of the video data. it saves data using block structure for fast accessing of patch video. This file uses a large space. -
frames_x_y folder
This folder saves the CNMF-E analysis for processing frames between x to y -
within the folder frames_x_y, there are folders LOGS_DATE corresponding to different runs of CNMF-E. and within each folder, there are two files intermediate_results.mat and logs.txt to save intermediate results and the log.
even though you set neuron.save_intermediate = false, you can still have a file intermediate_results.mat, but it should be much smaller. It only saves the results of the initialization step.
This design is useful when you want to load the initialization results of an earlier run.
If you don't want to keep any intermediate results, even the data_x_x_x.mat file, probably a better solution is to create another function neuron.clean_workspace() to remove these extra files. what do you think? @bahanonu
from cnmf_e.
Awesome, thanks for the description, might be good to have that as it's own wiki page (e.g. CNMF-E files created
) for future users.
A neuron.clean_workspace()
method sounds like a good solution, so users can have control over clearing the workspace themselves and can do their own checks (in terms of CNMF-E output being close what they expect) before removing. If it could get ride of the entire ROOTFOLDER_source_extraction
folder that is created, that would be helpful.
I ran with neuron.getReady_batch
using a function built around https://github.com/zhoupc/CNMF_E/edit/master/demos/demo_batch_1p.m, but I am still getting the 1. data_x_x_x.mat
file and frames_x_y
folder.
from cnmf_e.
Another solution is to allow the user access to dir_output
in https://github.com/zhoupc/CNMF_E/blob/master/ca_source_extraction/endoscope/distribute_data.m or an intermediate_folder_path
for anywhere else that an intermediate file is created so users can just put them all in a separate temporary folder or drive that can be cleared later.
from cnmf_e.
Can you update to allow the user to specify dir_output
? There are cases in which I would like to manually enter the location else run into the 260 character path limit on Windows, which causes CNMF-E to crash.
from cnmf_e.
Related Issues (20)
- CVX requirement doesn't work for Matlab 2017a HOT 2
- Typographic error in detrend data HOT 1
- Undefined function or variable 'A_prev'. HOT 2
- Error using save HOT 1
- The consuct "Ysignal(...end....)" is ambiguous HOT 3
- Error in updating brackgrand set in demo_large_data_1p.m HOT 1
- Error updating temporal components
- Error using save
- error in deconvolution HOT 1
- Bugfix in Sources2D save_workspace method and get_data_dimension.m HOT 5
- CNMF-E errors on movies with NaNs HOT 2
- error in update_spatial_parallel HOT 1
- Error in Sources2D/initComponents_parallel HOT 3
- CNMF-E errors on movies with NaNs, take 2 HOT 1
- Possibly use ndSparse for A matrices to reduce memory where there are lots of neurons HOT 2
- Error in demo 1p large data, update spatial parallel - zero sparse A column HOT 3
- Spike Count Estimate S HOT 2
- Source 2D issue
- Reconstruct neuron.C_raw from neuron.A
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cnmf_e.