draftcode/fastcsv

fastcsv's Issues

Python 3 compatibility

Hi,

while pip install fastcsv works seemlessly on my Python 2.7x installation on Ubuntu 14.04 (with gcc and python-dev installed), I cannot manage to install it on Python 3.5.1. The installation process almost immediately fails with the following output:

Collecting fastcsv
  Using cached fastcsv-0.1.2.tar.gz
Building wheels for collected packages: fastcsv
  Running setup.py bdist_wheel for fastcsv ... error
  Complete output from command /usr/local/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ncqqpon6/fastcsv/setup.py';exec(compile(getattr(tokenize, 'open', open)(__f
ile__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmp82902pxipip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.5
  copying fastcsv.py -> build/lib.linux-x86_64-3.5
  running build_ext
  building '_fastcsv' extension
  creating build/temp.linux-x86_64-3.5
  gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.5m -c _fastcsv.c -o build/temp.linux-x
86_64-3.5/_fastcsv.o
  _fastcsv.c: In function 'init_fastcsv':
  _fastcsv.c:37:38: warning: 'return' with no value, in function returning non-void [-Wreturn-type]
     if (PyType_Ready(&ReaderType) < 0) return;
                                        ^
  _fastcsv.c:38:38: warning: 'return' with no value, in function returning non-void [-Wreturn-type]
     if (PyType_Ready(&WriterType) < 0) return;
                                        ^
  _fastcsv.c:42:5: warning: implicit declaration of function 'Py_InitModule' [-Wimplicit-function-declaration]
       m = Py_InitModule("_fastcsv", _fastcsv_methods);
       ^
  _fastcsv.c:42:7: warning: assignment makes pointer from integer without a cast [enabled by default]
       m = Py_InitModule("_fastcsv", _fastcsv_methods);
         ^
  gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.5m -c _fastcsv_reader.c -o build/temp.
linux-x86_64-3.5/_fastcsv_reader.o
  _fastcsv_reader.c: In function 'ParseNewlineMode':
  _fastcsv_reader.c:57:5: warning: implicit declaration of function 'PyString_Check' [-Wimplicit-function-declaration]
       if (PyString_Check(newline)) {
       ^
  _fastcsv_reader.c: In function 'Reader_init':
  _fastcsv_reader.c:112:3: warning: implicit declaration of function 'PyString_FromString' [-Wimplicit-function-declaration]
     self->read_string = PyString_FromString("read");
     ^
  _fastcsv_reader.c:112:21: warning: assignment makes pointer from integer without a cast [enabled by default]
     self->read_string = PyString_FromString("read");
                       ^
  _fastcsv_reader.c:114:3: warning: implicit declaration of function 'PyInt_FromLong' [-Wimplicit-function-declaration]
     self->read_arg = PyInt_FromLong(1024);
     ^
  _fastcsv_reader.c:114:18: warning: assignment makes pointer from integer without a cast [enabled by default]
     self->read_arg = PyInt_FromLong(1024);
                    ^
  _fastcsv_reader.c: At top level:
  _fastcsv_reader.c:505:3: warning: missing braces around initializer [-Wmissing-braces]
     PyObject_HEAD_INIT(NULL)
     ^
  _fastcsv_reader.c:505:3: warning: (near initialization for 'ReaderType.ob_base.ob_base') [-Wmissing-braces]
  _fastcsv_reader.c:507:3: warning: initialization makes integer from pointer without a cast [enabled by default]
     "_fastcsv.Reader",             /* tp_name */
     ^
  _fastcsv_reader.c:507:3: warning: (near initialization for 'ReaderType.tp_basicsize') [enabled by default]
  _fastcsv_reader.c:510:3: warning: initialization from incompatible pointer type [enabled by default]
     (destructor)Reader_dealloc,    /* tp_dealloc */
     ^
  _fastcsv_reader.c:510:3: warning: (near initialization for 'ReaderType.tp_print') [enabled by default]
  _fastcsv_reader.c:525:24: error: 'Py_TPFLAGS_HAVE_ITER' undeclared here (not in a function)
     Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_ITER, /* tp_flags */
                          ^
  _fastcsv_reader.c:526:3: warning: initialization from incompatible pointer type [enabled by default]
     "FastCSV Reader Object",       /* tp_doc */
     ^
  _fastcsv_reader.c:526:3: warning: (near initialization for 'ReaderType.tp_traverse') [enabled by default]
  _fastcsv_reader.c:532:3: warning: initialization from incompatible pointer type [enabled by default]
     (iternextfunc)Reader_iternext, /* tp_iternext */
     ^
  _fastcsv_reader.c:532:3: warning: (near initialization for 'ReaderType.tp_methods') [enabled by default]
  _fastcsv_reader.c:533:3: warning: initialization from incompatible pointer type [enabled by default]
     Reader_methods,                /* tp_methods */
     ^
  _fastcsv_reader.c:533:3: warning: (near initialization for 'ReaderType.tp_members') [enabled by default]
  _fastcsv_reader.c:541:3: warning: initialization from incompatible pointer type [enabled by default]
     (initproc)Reader_init,         /* tp_init */
     ^
  _fastcsv_reader.c:541:3: warning: (near initialization for 'ReaderType.tp_alloc') [enabled by default]
  _fastcsv_reader.c:543:3: warning: initialization from incompatible pointer type [enabled by default]
     PyType_GenericNew,             /* tp_new */
     ^
  _fastcsv_reader.c:543:3: warning: (near initialization for 'ReaderType.tp_free') [enabled by default]
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for fastcsv
  Running setup.py clean for fastcsv
Failed to build fastcsv
Installing collected packages: fastcsv
  Running setup.py install for fastcsv ... error
    Complete output from command /usr/local/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ncqqpon6/fastcsv/setup.py';exec(compile(getattr(tokenize, 'open', open)(_
_file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-vqqcl5h3-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.5
    copying fastcsv.py -> build/lib.linux-x86_64-3.5
    running build_ext
    building '_fastcsv' extension
    creating build/temp.linux-x86_64-3.5
    gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.5m -c _fastcsv.c -o build/temp.linux
-x86_64-3.5/_fastcsv.o
    _fastcsv.c: In function 'init_fastcsv':
    _fastcsv.c:37:38: warning: 'return' with no value, in function returning non-void [-Wreturn-type]
       if (PyType_Ready(&ReaderType) < 0) return;
                                          ^
    _fastcsv.c:38:38: warning: 'return' with no value, in function returning non-void [-Wreturn-type]
       if (PyType_Ready(&WriterType) < 0) return;
                                          ^
    _fastcsv.c:42:5: warning: implicit declaration of function 'Py_InitModule' [-Wimplicit-function-declaration]
         m = Py_InitModule("_fastcsv", _fastcsv_methods);
         ^
    _fastcsv.c:42:7: warning: assignment makes pointer from integer without a cast [enabled by default]
         m = Py_InitModule("_fastcsv", _fastcsv_methods);
           ^
    gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.5m -c _fastcsv_reader.c -o build/tem
p.linux-x86_64-3.5/_fastcsv_reader.o
    _fastcsv_reader.c: In function 'ParseNewlineMode':
    _fastcsv_reader.c:57:5: warning: implicit declaration of function 'PyString_Check' [-Wimplicit-function-declaration]
         if (PyString_Check(newline)) {
         ^
    _fastcsv_reader.c: In function 'Reader_init':
    _fastcsv_reader.c:112:3: warning: implicit declaration of function 'PyString_FromString' [-Wimplicit-function-declaration]
       self->read_string = PyString_FromString("read");
       ^
    _fastcsv_reader.c:112:21: warning: assignment makes pointer from integer without a cast [enabled by default]
       self->read_string = PyString_FromString("read");
                         ^
    _fastcsv_reader.c:114:3: warning: implicit declaration of function 'PyInt_FromLong' [-Wimplicit-function-declaration]
       self->read_arg = PyInt_FromLong(1024);
       ^
    _fastcsv_reader.c:114:18: warning: assignment makes pointer from integer without a cast [enabled by default]
       self->read_arg = PyInt_FromLong(1024);
                      ^
    _fastcsv_reader.c: At top level:
    _fastcsv_reader.c:505:3: warning: missing braces around initializer [-Wmissing-braces]
       PyObject_HEAD_INIT(NULL)
       ^
    _fastcsv_reader.c:505:3: warning: (near initialization for 'ReaderType.ob_base.ob_base') [-Wmissing-braces]
    _fastcsv_reader.c:507:3: warning: initialization makes integer from pointer without a cast [enabled by default]
       "_fastcsv.Reader",             /* tp_name */
       ^
    _fastcsv_reader.c:507:3: warning: (near initialization for 'ReaderType.tp_basicsize') [enabled by default]
    _fastcsv_reader.c:510:3: warning: initialization from incompatible pointer type [enabled by default]
       (destructor)Reader_dealloc,    /* tp_dealloc */
       ^
    _fastcsv_reader.c:510:3: warning: (near initialization for 'ReaderType.tp_print') [enabled by default]
    _fastcsv_reader.c:525:24: error: 'Py_TPFLAGS_HAVE_ITER' undeclared here (not in a function)
       Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_ITER, /* tp_flags */
                            ^
    _fastcsv_reader.c:526:3: warning: initialization from incompatible pointer type [enabled by default]
       "FastCSV Reader Object",       /* tp_doc */
       ^
    _fastcsv_reader.c:526:3: warning: (near initialization for 'ReaderType.tp_traverse') [enabled by default]
    _fastcsv_reader.c:532:3: warning: initialization from incompatible pointer type [enabled by default]
       (iternextfunc)Reader_iternext, /* tp_iternext */
       ^
    _fastcsv_reader.c:532:3: warning: (near initialization for 'ReaderType.tp_methods') [enabled by default]
    _fastcsv_reader.c:533:3: warning: initialization from incompatible pointer type [enabled by default]
       Reader_methods,                /* tp_methods */
       ^
    _fastcsv_reader.c:533:3: warning: (near initialization for 'ReaderType.tp_members') [enabled by default]
    _fastcsv_reader.c:541:3: warning: initialization from incompatible pointer type [enabled by default]
       (initproc)Reader_init,         /* tp_init */
       ^
    _fastcsv_reader.c:541:3: warning: (near initialization for 'ReaderType.tp_alloc') [enabled by default]
    _fastcsv_reader.c:543:3: warning: initialization from incompatible pointer type [enabled by default]
       PyType_GenericNew,             /* tp_new */
       ^
    _fastcsv_reader.c:543:3: warning: (near initialization for 'ReaderType.tp_free') [enabled by default]
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/local/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ncqqpon6/fastcsv/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace(
'\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-vqqcl5h3-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-ncqqp
on6/fastcsv/

Is there any possibility to also install the library on Python 3.x?

Fail to detect lineendings when they are on the edge of the buffer

Universal newline mode of fastcsv fails to detect CRLF if CR is at the end of cached buffer and the next unread char is LF.

Segmentation Fault when fileobj.close() raises an error

Best practices for handing tab delimited files using fastcsv?

I have many large tab seperated files which I need to parse with python. I found fastcsv and it seems like what I need, except for using commas.

Is there an efficient way to wrap an io.open stream to replace tabs with commas? I'm not familiar with the API.

It raises IOError when it reads a non-quoted cell that contains over 1024 chars.

ValueError raised on files with some columns quoted, others not

Running

with fastcsv.Reader(io.open("/home/zimmi/documents/clevermaps/cm-temp/test.csv", encoding="ISO-8859-2")) as reader:
    for row in reader:
        print row

results in ValueError: string before quote for a csv file with a single row one;"two".

It works just fine with builtin csv module.
Any chance to make this work?

Extra flush call will write buffer twice.

writer = fastcsv.Writer(out)
writer.writerow([None])
writer.flush()
writer.flush()

This code will write out ""\r\n""\r\n