Some IOs cannot handle multi-segment blocks and fail tests

[apdavison]

Some IOs cannot handle multi-segment blocks and fail tests

[apdavison]

crodger wrote on 2011-10-04 22:51:55:

Text of error:

{{{

FAIL: test_write_then_read (neo.test.io.test_pynnio.CommonTestPyNNTextIO)

Traceback (most recent call last):
File "/home/chris/neo2_dev/neo0.2/neo/test/io/common_io_test.py", line 200, in test_write_then_read
assert_same_sub_schema(ob, ob2)
File "/home/chris/neo2_dev/neo0.2/neo/test/tools.py", line 128, in assert_same_sub_schema
assert len(sub1) == len(sub2), 'theses two %s have not the same %s number'%(classname, child)
AssertionError: theses two Segment have not the same SpikeTrain number

}}}

[apdavison]

crodger wrote on 2011-10-05 00:49:12:

Sorry, change (3) above doesn't make any sense and I need to clarify it. We don't need to add Block.annotations['segment_times'] because this information is already in Block.segments. The problem is that, once it is written to disk, the segment tstarts are lost. What I am suggesting is a new CSV file with segment info that allows the segments to be reconstructed after reading from disk.

[apdavison]

sgarcia wrote on 2011-10-05 10:58:13:

I think we should had a flag in BaseTestIO,
write_them_read_maintain_schema = True or False
and the test is skipped or not.

So for IO for which read/write is bijectif the test would be avoid.

Note: I do not like so much an IO that automagically concatenate 3 Segment in 1 Segment because they are continuous. We have the same cases in AxonIO, several continuous Segment and in that case I create 3 Segment I do not concatenate them. I think It is more transparent for the user. If you do a concatenate, I tink you should add a flag MyIO.read_block(concatenate_segment_if_continuous = True).

[apdavison]

crodger wrote on 2011-10-06 01:36:28:

So, do you write each segment to a separate file? I don't think the Blackrock format includes the datetime in the file itself, but in a separate metadata file, which I don't know how to parse.

[apdavison]

sgarcia wrote on 2011-10-06 10:30:58:

I am not sure to understand you. You read a file that is in fact 3 segments, at reading times you concatenate then in one because they are continuous so you return 1 segment. So at writing time, you write 1 segment, because you have only that. Am I wrong ?

In my case I do not write. I just render whil reading 3 segments when there 3 segments in the files. Even if they are continuous and they could be concatenated. In Axon software you can concatenate several files in one. It is typically 1 block - 3 segment cases.

[apdavison]

sgarcia wrote on 2011-10-28 15:14:40:

I made a patch to skip this test in Blackrock.

So this ticket is closed but I have a new ticket ticket for neo 0.3 because the way BlackrockIO is implemented (multi segment of one same segment for getting chunk of data) cause the problem.