⏳🛡 Pythonic universal errors-and-erasures Reed-Solomon codec to protect your data from errors and bitrot. Includes a future-proof zero-dependencies pure-python implementation 🔮 and an optional speed-optimized Cython/C extension 🚀
Home Page: http://pypi.python.org/pypi/reedsolo
License: Other
Also, I wonder if there is an error in line 681 (of reedsolo.py)
errs = len(err_loc) - 1
why the -1?
Thanks to Jonas Braun for reporting this possible issue!
Mirror of lrq3000#1
I think it's a bug, but I'm not 100% sure, because I'm playing with RS codec for just few hours.
>>> import reedsolo
>>> c = reedsolo.RSCodec(nsym=0, nsize=1)
>>> c.encode(b'aaa')
bytearray(b'aaa')
>>> c.decode(b'aaa')
bytearray(b'')
prim=285 is the same, as the length and number of correction symbols.
Why is the difference?
How can I get same result?
Mirror of lrq3000#4
Curl has experienced the same issue: curl/curl#7150
Need to find an alternative. GitHub Continuous Integration maybe?
Hello!
I'm having issues with using the cythonized creedsolo, encoding and checking seems to work great, but my attempt to correct errors results in a ReedSolomonError:
FAILED test.py::test_reed_solomon - creedsolo.creedsolo.ReedSolomonError: Too many (or few) errors found by Chien Search for the errata locator polynomial!
(It works as expected with the pure Python version.)
My test case used:
# Pure Python works as expected
# import reedsolo as rs
import creedsolo as rs
DATA = b'123456789abcdef\n' * 64
def test_reed_solomon():
prim = rs.find_prime_polys(c_exp=11, fast_primes=True, single=True)[0]
rs.init_tables(c_exp=11, prim=prim)
gen = rs.rs_generator_poly_all(1024+16)
encoded = rs.rs_encode_msg(DATA, 16, gen=gen[16])
encoded[3] = ord('7')
encoded[100] = ord('!')
assert not rs.rs_check(encoded, 16)
_, _, errata_pos = rs.rs_correct_msg(encoded, 16)
assert set(errata_pos) == {3, 100}FWIW, I used the current Git master @ 23b8eab
pip install . --config-setting="--build-option=--cythonize"
Hi, I'm trying to use this module to verify the so-called "mode message" of Aztec codes (see: https://en.wikipedia.org/wiki/Aztec_Code#Mode_message)
Reed Solomon is used on GF(16), using the polynomial x^4 + x + 1 (= 0x13, 10011b, 19 decimal). The message is composed of 7 words, two 4-bit data words followed by five 4-bit ECC words.
The reference Aztec code from the specification has the following mode message: [0, 9, 12, 2, 3, 1, 9] (0000100111000010001100011001 in binary), which I just can't get to decode, and I don't understand why.
My verification code:
>>> from reedsolo import RSCodec, ReedSolomonError
>>> words = [0, 9, 12, 2, 3, 1, 9]
>>> prim = 19 # 10011 = x^4 + x^1 + x^0
>>> c_exp = 4 # 2^4 = GF(16)
>>> nsym = 5 # number of ECC words for compat Aztec codes
>>> rsc = RSCodec(nsym=nsym, c_exp=c_exp, prim=prim)
>>> rsc.decode(words)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File ".../venv/lib/python3.10/site-packages/reedsolo.py", line 929, in decode
rmes, recc, errata_pos = rs_correct_msg(chunk, nsym, fcr=self.fcr, generator=self.generator, erase_pos=e_pos, only_erasures=only_erasures)
File ".../venv/lib/python3.10/site-packages/reedsolo.py", line 760, in rs_correct_msg
raise ReedSolomonError("Could not correct message")
reedsolo.ReedSolomonError: Could not correct message
I also checked another implementation (zxing) and they use the same RS parameters as I do ( https://github.com/zxing-cpp/zxing-cpp/blob/master/core/src/GenericGF.cpp#L33), and there the same message decodes correctly there (0 errors).
Can someone tell me where I'm wrong? I just started digging into Reed-Solomon codes so it might very well be a fundamental misunderstanding on my side. :-)
This is more a feature request. I have a RS encoding that is a bit strange. It uses 8 bytes of error correction, but instead of the expected encoding:
rs8a=reedsolo.RSCodec(nsym=8, fcr=0, prim=0x11d, c_exp=8)
it instead encodes for 16 bytes, but omits the last 8 bytes from the message.
Due to the way that reed-solomon can deal with known erasures it has the same correcting capabilities as the "standard" one.
I have written a small helper function to deal with this case:
# data: 31B, checksum: 8B, erasure: 8B - RS(47,31) - transmission omits last 8B
rs8=reedsolo.RSCodec(nsym=(8+8), fcr=0, prim=0x11d, c_exp=8)
rs8.elen=8
def decode_with_fixed_erasure(self, data, nsym=None, erase_pos=None, only_erasures=False):
if nsym is None: nsym=self.nsym
if self.elen > 0:
data=data+([0]*self.elen)
r=range(len(data)-self.elen,len(data))
if erase_pos is None:
erase_pos=r
else:
erase_pos=list(set(r)|set(erase_pos))
(cmsg,crs,ep)=self.decode(data, nsym, erase_pos=erase_pos, only_erasures=only_erasures)
return (cmsg, crs, bytearray(set(ep)-set(r)))
else:
return self.decode(data, nsym, erase_pos, only_erasures)
I wonder if this is something you would consider adding support for.
No clue what's happening here. 1.7.0 cythonizes just fine. I've tried it in venv too.
$ cython --version
Cython version 0.29.33
$ python --version
Python 3.11.2
$ python setup.py build --cythonize
Cython is installed, building creedsolo module
[1/1] Cythonizing creedsolo.pyx
Error compiling Cython file:
------------------------------------------------------------
...
^
------------------------------------------------------------
cython:0:0: cython.cimports is not available
Traceback (most recent call last):
File "/tmp/reedsolomon/setup.py", line 32, in <module>
extensions = cythonize([ Extension('creedsolo', ['creedsolo.pyx']) ], annotate=True, force=True, # this may fail hard if Cython is installed but there is no C compiler for current Python version, and we have no way to know. Alternatively, we could supply exclude_failures=True , but then for those who really want the cythonized compiled extension, it would be much harder to debug
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1117, in cythonize
cythonize_one(*args)
File "/usr/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1240, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: creedsolo.pyx
I'm not sure if it's just me.
But I first found this issue with the wikiversity article, so I thought the python reed-solo wouldn't have this issue, but I'm encountering the same issue.
msg_in = [ 0x40, 0xd2, 0x75, 0x47, 0x76, 0x17, 0x32, 0x06, 0x27, 0x26, 0x96, 0xc6, 0xc6, 0x96, 0x70, 0xec ]
msg = rs_encode_msg(msg_in, 10)
synd = rs_calc_syndromes(msg, 10)
print("Message initial: ")
for i in range(0,len(msg)):
print(hex(msg[i]), end=' ')
print(synd) # not corrupted message = all 0 syndromes
msg[0] = 0 # deliberately damage the message
synd = rs_calc_syndromes(msg, 10)
print("Message after: ")
for i in range(0,len(msg)):
print(hex(msg[i]), end=' ')
print(synd) # when corrupted, the syndromes will be non zero
synd = rs_calc_syndromes(msg, 10)
msg = rs_correct_errata(msg, synd, [0]) # [0] is the list of the erasures locations, here it's the first character, at position 0
print(hex(msg[0]))
After injecting error by changing the first bit of the message into zero, I recalculate the syndromes, but the resulting syndrome is exactly the same! Meaning rs_correct_errata makes no correction.
Could someone let me know whether I'm just not understanding.
Hi, I am trying to install esptool and reedsolo is a dependency however installation with pip fails with the following error message
Building wheels for collected packages: reedsolo
Building wheel for reedsolo (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [124 lines of output]
Cython is installed, building creedsolo module
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
copying reedsolo.py -> build/lib.linux-x86_64-cpython-310
running build_ext
building 'creedsolo' extension
creating build/temp.linux-x86_64-cpython-310
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/usr/include/python3.10 -c creedsolo.c -o build/temp.linux-x86_64-cpython-310/creedsolo.o
creedsolo.c: In function ‘__pyx_tp_dealloc_array’:
creedsolo.c:32027:5: error: lvalue required as increment operand
32027 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32029:5: error: lvalue required as decrement operand
32029 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc_memoryview’:
creedsolo.c:32338:5: error: lvalue required as increment operand
32338 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32340:5: error: lvalue required as decrement operand
32340 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
creedsolo.c:32588:5: error: lvalue required as increment operand
32588 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32590:5: error: lvalue required as decrement operand
32590 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__Pyx_InitGlobals’:
creedsolo.c:33787:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations]
33787 | PyEval_InitThreads();
| ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:130,
from creedsolo.c:16:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_ParseOptionalKeywords’:
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_decode_c_string’:
creedsolo.c:37955:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
37955 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for reedsolo
Running setup.py clean for reedsolo
Failed to build reedsolo
Installing collected packages: reedsolo, bitstring, ecdsa, esptool
Running setup.py install for reedsolo ... error
error: subprocess-exited-with-error
× Running setup.py install for reedsolo did not run successfully.
│ exit code: 1
╰─> [126 lines of output]
Cython is installed, building creedsolo module
running install
/home/sbn/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
copying reedsolo.py -> build/lib.linux-x86_64-cpython-310
running build_ext
building 'creedsolo' extension
creating build/temp.linux-x86_64-cpython-310
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/usr/include/python3.10 -c creedsolo.c -o build/temp.linux-x86_64-cpython-310/creedsolo.o
creedsolo.c: In function ‘__pyx_tp_dealloc_array’:
creedsolo.c:32027:5: error: lvalue required as increment operand
32027 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32029:5: error: lvalue required as decrement operand
32029 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc_memoryview’:
creedsolo.c:32338:5: error: lvalue required as increment operand
32338 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32340:5: error: lvalue required as decrement operand
32340 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
creedsolo.c:32588:5: error: lvalue required as increment operand
32588 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32590:5: error: lvalue required as decrement operand
32590 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__Pyx_InitGlobals’:
creedsolo.c:33787:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations]
33787 | PyEval_InitThreads();
| ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:130,
from creedsolo.c:16:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_ParseOptionalKeywords’:
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_decode_c_string’:
creedsolo.c:37955:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
37955 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> reedsolo
Please help me solve this issue.
Thanks
example code:
import reedsolo
rsc = reedsolo.RSCodec(12, c_exp=12) # same as nsize=4095
str_msg = "This is a message"
bytes_msg = b"This is a binary message"
breakpoint()
result = rsc.encode(str_msg) # this works
result_b = rsc.encode(bytes_msg) # this fails
Error message:
Traceback (most recent call last):
File "rs_tensor.py", line 120, in <module>
result_b = rsc.encode(bytes_msg) # this fails
File "reedsolo.py", line 893, in encode
enc.extend(rs_encode_msg(chunk, self.nsym, fcr=self.fcr, generator=self.generator, gen=self.gen[nsym]))
File "reedsolo.py", line 526, in rs_encode_msg
lcoef = gf_log[coef] # precaching
IndexError: array index out of range
The problem seems to stem from the fact that the code in _bytearray has the following fall-through code (if obj is not a str or int)
278 # Else obj is a list of int, it's ok
279 return array("i", obj)
When in reality, obj is of type bytes.
Proposed fix:
def _bytearray(obj = 0, encoding = "latin-1"):
'''Fake bytearray replacement, supporting int values above 255'''
# always use Latin-1 and not UTF8 because Latin-1 maps the first 256 characters to their bytevalue equivalents. UTF8 may mangle your data (particularly at vale 128)
if isinstance(obj, str): # obj is a string, convert to list of ints
obj = obj.encode(encoding)
if isinstance(obj, str): # Py2 str: convert to list of ascii ints
obj = [ord(chr) for chr in obj]
elif isinstance(obj, bytes): # Py3 bytes: characters are bytes, need to convert to int for array.array('i', obj)
obj = [int(chr) for chr in obj]
else:
raise(ValueError, "Type of object not recognized!")
elif isinstance(obj, int): # compatibility with list preallocation bytearray(int)
obj = [0] * obj
elif isinstance(obj, bytes):
obj = [int(b) for b in obj]
# Else obj is a list of int, it's ok
return array("i", obj)
I follow the steps described in the read me to install the cython version, with no luck. Specifically:
jcea@jcea:/tmp/ram$ pip install --upgrade reedsolo --install-option="--cythonize" --verbose
Usage:
pip install [options] <requirement specifier> [package-index-options] ...
pip install [options] -r <requirements file> [package-index-options] ...
pip install [options] [-e] <vcs project url> ...
pip install [options] [-e] <local project path> ...
pip install [options] <archive url/path> ...
no such option: --install-option
Python 3.11.3
pip 23.1.2
cython 0.29.34
I'd like to use some of the newer features, namely the error_pos argument to RSCodec.decode. It seems like the latest version in pypi is 0.3. Is it possible to get the latest version pushed?
It may be result of latest cython upgrade
+ /usr/bin/python3 -sBm build -w --no-isolation -C=--build-option=--cythonize
* Getting build dependencies for wheel...
running egg_info
creating reedsolo.egg-info
writing reedsolo.egg-info/PKG-INFO
writing dependency_links to reedsolo.egg-info/dependency_links.txt
writing top-level names to reedsolo.egg-info/top_level.txt
writing manifest file 'reedsolo.egg-info/SOURCES.txt'
reading manifest file 'reedsolo.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'reedsolo.egg-info/SOURCES.txt'
* Building wheel...
Cython is installed, building creedsolo module
[1/1] Cythonizing creedsolo.pyx
Error compiling Cython file:
------------------------------------------------------------
...
^
------------------------------------------------------------
cython:0:0: cython.cimports is not available
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 413, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 398, in _build_with_temp_dir
self.run_setup()
File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 484, in run_setup
super(_BuildMetaLegacyBackend,
File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
exec(code, locals())
File "<string>", line 32, in <module>
File "/usr/lib64/python3.8/site-packages/Cython/Build/Dependencies.py", line 1115, in cythonize
cythonize_one(*args)
File "/usr/lib64/python3.8/site-packages/Cython/Build/Dependencies.py", line 1238, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: creedsolo.pyx
ERROR Backend subprocess exited when trying to invoke build_wheelHere is list of installed modules in build env
Package Version
--------------- -------
build 0.10.0
Cython 0.29.34
distro 1.8.0
exceptiongroup 1.0.0
gpg 1.19.0
iniconfig 2.0.0
installer 0.7.0
libcomps 0.1.19
packaging 23.0
pluggy 1.0.0
pyproject_hooks 1.0.0
pytest 7.3.0
python-dateutil 2.8.2
setuptools 65.6.3
six 1.16.0
tomli 2.0.1
wheel 0.40.0Hi there!
I'm currently exploring ways to achieve a similar result as the reed-solomon Go library in Python. In the Go library, the encoding process operates on a double array of bytes, which is then passed through the codeSomeShards function to create parity shards.
I'm wondering if it's possible to obtain a similar outcome using Python.
Any suggestions or insights would be greatly appreciated!
This code:
import reedsolo
reedsolo.RSCodec(300).encode(b'a')
Returns bytearray(b'') on my computer. Adjusting the number down causes it to return something.
Hi,
I found a bug in the decoding part. It is good in Windows. In Linux and Mac OS, we found the decoder return a list [bytearray 1, bytearray 2, bytearray 3] not just a bytearray. The bytearray 1 is what we need in the decoding part. However, I am not sure why it is different in different operating systems.
In my package, we do the follow sentence to fix this problem (TypeError: 'bytearray' object cannot be interpreted as an integer):
self.tool = RSCodec(3)
decode_byte_list = list(self.tool.decode(byte_list))
if type(decode_byte_list[0]) is not int:
decode_byte_list = list(decode_byte_list[0])Best and FYI,
Zhang
How can I setup it with Python 3.9?
Hi,
Maybe you can extend the api so that decode() returns a tuple:
(data, n_errors_seen)
?
That way one can anticipate by lowering baudrates and such.
I have code that needs to verify two different reedsolo checksums at different times.
one with c_exp=8, the second one with c_exp=6.
My current solution is to copy reedsolo.py to reedsolo6.py and import both reedsolo and reedsolo6 so i can have two instances.
It would be nice if this were supported out of the box :-)
So, say I want to use Reed Solomon such as the Voyager Code (255, 223). How would I do that?
From your Readme:
The codec has quite reasonable performances if you either use PyPy on the pure-python implementation (reedsolo.py) or either if you compile the Cython extension creedsolo.pyx (which is about 2x faster than PyPy). You can expect encoding rates of several MB/s.
Is this still valid?
I just did a performance evaluation for three different python versions (3.7, 3.8, 3.9) with the following code.
import sys
import numpy as np
from reedsolo import RSCodec
import perfplot
name = f'perf_v{sys.version_info.major}.{sys.version_info.minor}'
def func(rscoder, array):
enc = rscoder.encode(array)
return rscoder.decode(enc)[0]
codecs = [
RSCodec(8),
RSCodec(16),
]
out = perfplot.bench(
setup = lambda n: np.random.randint(0,255,size=n, dtype=np.uint8),
kernels = [
lambda a: func(codec, a) for codec in codecs
],
labels = [codec.nsym for codec in codecs],
n_range = [2 ** k for k in range(20)],
)
out.show()
out.save(name + ".png", transparent=True, bbox_inches="tight")
I get a maximum of 100kB/s (for encoding and decoding together).
Is this an expected speed? Tested on Ubuntu 20.04, cython is installed.
Here is the output of the three perfplots.
I expected the cythonized function to be faster. However the number of ecc symobls seems to be not relevant here.
PS:
To reproduce with python3.7, you'll have to install perfplot==0.9.6
Is there a way of getting the Systematic Generator Matrix, actually Parity Matrix from the Generator Matrix from the field generator polynomial. So if I give my own field generator polynomial, can the lib return the generator matrix?
Looks like something is wrog when pep517 is used and DSO modules is not build
+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
running egg_info
creating reedsolo.egg-info
writing reedsolo.egg-info/PKG-INFO
writing dependency_links to reedsolo.egg-info/dependency_links.txt
writing top-level names to reedsolo.egg-info/top_level.txt
writing manifest file 'reedsolo.egg-info/SOURCES.txt'
reading manifest file 'reedsolo.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'reedsolo.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib
copying reedsolo.py -> build/lib
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
copying build/lib/reedsolo.py -> build/bdist.linux-x86_64/wheel
running install_egg_info
running egg_info
writing reedsolo.egg-info/PKG-INFO
writing dependency_links to reedsolo.egg-info/dependency_links.txt
writing top-level names to reedsolo.egg-info/top_level.txt
reading manifest file 'reedsolo.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'reedsolo.egg-info/SOURCES.txt'
Copying reedsolo.egg-info to build/bdist.linux-x86_64/wheel/reedsolo-1.7.0-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/reedsolo-1.7.0.dist-info/WHEEL
creating '/home/tkloczko/rpmbuild/BUILD/reedsolomon-1.7.0/dist/.tmp-h1l51r67/reedsolo-1.7.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'reedsolo.py'
adding 'reedsolo-1.7.0.dist-info/LICENSE'
adding 'reedsolo-1.7.0.dist-info/METADATA'
adding 'reedsolo-1.7.0.dist-info/WHEEL'
adding 'reedsolo-1.7.0.dist-info/top_level.txt'
adding 'reedsolo-1.7.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built reedsolo-1.7.0-py3-none-any.whlI tried to use cython and installed with
sudo python3 setup.py install --native-compile
Installation seem to have worked but I get this error calling the check method:
File "creedsolo.pyx", line 870, in creedsolo.RSCodec.check
AttributeError: 'RSCodec' object has no attribute 'chunk'
Am I doing something wrong?
Could you please push the tag for 1.7.0 release? We use tags to make Gentoo packages here.
Hey buddy, as is said in the mail, I tried to improve this package, but encountered an error in the decoder. The improvements to the encoder may help, so I send it to you, hoping that you (or some one else) can (easily) fix it.
p.s. Source code is not permitted to be attached here.
Hi,
I would like to use an alphabet of 2^5 possible symbols (base32 encoding) for my message and code. I tried setting c_exp to 5 and get the following error:
>>> rs = reedsolo.RSCodec(nsym=2, nsize=8, c_exp=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/dist-packages/reedsolo-0.5-py3.4-linux-x86_64.egg/reedsolo.py", line 756, in __init__
init_tables(prim, generator, c_exp)
File "/usr/local/lib/python3.4/dist-packages/reedsolo-0.5-py3.4-linux-x86_64.egg/reedsolo.py", line 201, in init_tables
gf_exp[i] = x # compute anti-log for this value and store it in a table
ValueError: byte must be in range(0, 256)
I want to get something like a flight code (length 8 digits/characters) with error correction.
Has anyone tried using different symbols? Is it possible using the base library instead of the hight level API?
Thanks
The third element of the rs_correct_msg is a list of error positions. The positions returned are inconsistent. Consider the following:
>>> import reedsolo as rs
>>> from reedsolo import rs_encode_msg, rs_correct_msg, rs_check
>>> _ = rs.init_tables()
>>> msg = rs_encode_msg(bytes(range(10)), nsym=4)
>>> msg
bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\xf0\x9f\x84\xea')
>>> rs_correct_msg(msg, nsym=4, erase_pos=[1]) # CALL 1
(bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'), bytearray(b'\xf0\x9f\x84\xea'), [1])
>>> rs_correct_msg(msg, nsym=4, erase_pos=[0]) # CALL 2
(bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'), bytearray(b'\xf0\x9f\x84\xea'), [])
>>> msg[0] = 0xFF
>>> rs_correct_msg(msg, nsym=4) # CALL 3
(bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'), bytearray(b'\xf0\x9f\x84\xea'), [0])
>>> rs_correct_msg(msg, nsym=4, erase_pos=[0]) # CALL 4
(bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'), bytearray(b'\xf0\x9f\x84\xea'), [])
In call 1, the message is correct, but we mark position 1 as an erasure. The returned bytearrays match the input message, but rs_correct_msg reports position 1 as an error. This is defensible, as that byte was logically erased, even though the correct byte happened to be in the array. But then call 2 should report position 0 as an error.
We then corrupt byte 0 of the message. Call 3 corrects the corruption and reports position 0 as an error (which it is). In call 4 we mark position 0 as an erasure; this call also corrects the corruption, but does not report position 0 as an error.
I think the source of this inconsistency is reedsolo.py lines 728-729, which write 0 to each erased position. In call 1, this corrupts the message, which is subsequently corrected and the error position correctly returned; in call 2, the erased position is already zero, so the message remains correct and rs_correct_msg returns on line 780 with an empty error position list. In call 3, no zeroing happens because no erasures were given, so the message is corrected and the error position correctly returned; in call 4, the zeroing corrects the message and the call returns on line 780 with an empty error position list.
If you agree that all erased positions should be reported as error positions, and you want to keep the zeroing for ease of debugging (according to the comment), the easy fix is to return error_pos on line 780 (in place of the empty list).
In the License section of README.rst, you state:
If the Public Domain is not adequate for your purpose, you can instead consider this module under the MIT License as you prefer.
However, one of the provisions of the MIT license is that the copyright notice and license text shall be included in all copies. It is impossible to comply with this unless you add an original MIT license text with appropriate copyright notice—preferably in its own file, like LICENSE-MIT, or perhaps in an added section in the existing LICENSE file.
Please see this documentation from the Fedora Linux project for further explanation. (From the perspective of the Fedora Linux project, the Public Domain option is perfectly adequate.)
Thank you for your interest in or for using this module. Before @tomerfiliba made it 10 years ago and I took the mantle 7 years ago, error correction was something that was available only in proprietary softwares, or hidden in the entrails of academic works as side functions often mixed in with the core routines of these academic projects that were aimed at other purposes than pure error correction. This module was an experiment to provide an out-of-the-box readily useable yet multifunctional tool for general error correction, that could then be used for numerous practical applications, from QR code generators and readers, to hard drive data protection and recovery and of course data stream denoising.
While I am glad of the positive impact this project had, it was never intended to be used so intensively in production. It was made as an educational tool and as robust as possible proof of concept of a universal Reed-Solomon encoder/decoder, to show the ropes with most of the maths worked out and implemented in a working Python package to hopefully inspire others from making a better suite of tools, using this as a reference to check the maths.
But now this Python module is used as-is in numerous other projects (thousands from what GitHub shown me, and likely more given there are >300K monthly downloads!), that are relying on it for the very sensitive task of ensuring data preservation and reliability. This is a huge responsibility, and unfortunately there is no one maintaining this project but me, someone who is not even in the field of signal processing (I worked on some such projects, but this is not my specialty, this project was just a "fun" side project).
In the last 2 years, I hoped someone would jump in to replace this project with another more robust one, but it seems no one did. I will hence continue to try to maintain this project once in a while, very infrequently and with no guarantee, just to ensure the project still continues to work. But no new features will be implemented. PRs are very welcome to bug fixes, otherwise they may not ever be corrected.
I aim to maintain the software to run as-is, and it's already a lot of work given the fast pace of the Python ecosystem evolution. If anyone else wants to step in to maintain or even develop further the project (under open-source of course), please reply in this thread and I will consider your application.
Thank you for reading this message and thanks to the authors of this reed-solomon Python repository.
If we take (255, 223) coding as an example and assume GF(2^8), the information before coding has 223 bytes, and the reed-solomon ecc has 32 bytes. Hence, the total encoded length is 255 bytes.
First, the information before coding has a file named "input.bin", which has 233 bytes.
| input.bin | column 1 | column 2 | column 3 | column 4 | column 5 | column 6 | column 7 | column 8 | column 9 | column 10 | column 11 | column 12 | column 13 | column 14 | column 15 | column 16 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| row 1 | 0x12 | 0x34 | 0x56 | 0x78 | 0x90 | 0x11 | 0x22 | 0x33 | 0x44 | 0x55 | 0x66 | 0x77 | 0x88 | 0x99 | 0x00 | 0xff |
| row 2 | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff | 0xff |
| row 3 | 0x32 | 0x3a | 0x63 | 0x75 | 0x3c | 0x77 | 0xb3 | 0x83 | 0x86 | 0x86 | 0x5c | 0x00 | 0x00 | 0x20 | 0x00 | 0x00 |
| row 4 | 0x00 | 0x00 | 0x20 | 0x02 | 0x00 | 0x10 | 0x00 | 0x00 | 0x01 | 0x00 | 0x00 | 0x00 | 0xb0 | 0xf1 | 0x01 | 0x00 |
| row 5 | 0x00 | 0x00 | 0x60 | 0x08 | 0x00 | 0x00 | 0x18 | 0x30 | 0x97 | 0xe5 | 0x00 | 0x00 | 0x53 | 0xe3 | 0x0c | 0x00 |
| row 6 | 0x00 | 0x1a | 0xd8 | 0x28 | 0xcd | 0xe1 | 0x78 | 0x10 | 0x96 | 0xe5 | 0x06 | 0x00 | 0xa0 | 0xe1 | 0x31 | 0xff |
| row 7 | 0x2f | 0xe1 | 0x00 | 0x20 | 0x50 | 0xe2 | 0xe5 | 0xff | 0xff | 0xca | 0x05 | 0x00 | 0x00 | 0x0a | 0xcc | 0x01 |
| row 8 | 0x9f | 0xe5 | 0x0a | 0x10 | 0xa0 | 0xe1 | 0x00 | 0x00 | 0x8f | 0xe0 | 0xdc | 0xa0 | 0x90 | 0xe5 | 0x01 | 0x10 |
| row 9 | 0x41 | 0xe2 | 0x94 | 0x90 | 0x90 | 0xe5 | 0x01 | 0x20 | 0x02 | 0xe0 | 0x00 | 0x30 | 0xa0 | 0xe3 | 0xc8 | 0xd0 |
| row10 | 0x4d | 0xe2 | 0x03 | 0x00 | 0x92 | 0xe1 | 0x03 | 0x00 | 0x00 | 0x0a | 0x50 | 0x03 | 0x9f | 0xe5 | 0x00 | 0x00 |
| row11 | 0x8f | 0xe0 | 0xa0 | 0x90 | 0xe5 | 0x01 | 0x10 | 0x41 | 0xe2 | 0x94 | 0x90 | 0x90 | 0xe5 | 0x01 | 0x20 | 0x02 |
| row12 | 0xe0 | 0x00 | 0x30 | 0xa0 | 0xe3 | 0xc8 | 0xd0 | 0x4d | 0xe2 | 0x03 | 0x00 | 0x92 | 0xe1 | 0x03 | 0x08 | 0x10 |
| row13 | 0x9d | 0xe5 | 0x00 | 0x00 | 0x51 | 0xe3 | 0x04 | 0x00 | 0x00 | 0x0a | 0x08 | 0x20 | 0x9d | 0xe5 | 0x04 | 0x00 |
| row14 | 0xa0 | 0x08 | 0x10 | 0x9d | 0xe5 | 0x00 | 0x00 | 0x51 | 0xe3 | 0x04 | 0x00 | 0x00 | 0x0a | 0x08 | 0x20 | - |
Second, I use the MATLAB to generate the RS code, and the result of 32-byte RS code is below. Please note that this 32-byte RS code generated by MATLAB is my desired result. However, the Python script provided by his Github repository has different results (see the third part).
m = 8;
n = 255;
k = 223;
prim_poly = 285;
genpoly = rsgenpoly(n,k,prim_poly,1);
fp = fopen('input.bin','r','b') ;
data = fread(fp, 'ubit8')';
info_bytes = gf(data ,m);
encode_bytes = rsenc(info_bytes,n,k,genpoly) ;
encode_bytes.x %print this, then see the 255 bytes with rs code
| rs code | column 1 | column 2 | column 3 | column 4 | column 5 | column 6 | column 7 | column 8 | column 9 | column 10 | column 11 | column 12 | column 13 | column 14 | column 15 | column 16 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| row 1 | 0x28 | 0x49 | 0x62 | 0x92 | 0x1d | 0x89 | 0x5c | 0x81 | 0x02 | 0xde | 0x7b | 0x1f | 0xb3 | 0xc1 | 0xa8 | 0x48 |
| row 2 | 0xd4 | 0xe9 | 0xad | 0x47 | 0xf4 | 0x70 | 0x58 | 0x8b | 0x97 | 0x1d | 0x8a | 0x83 | 0x42 | 0x32 | 0x29 | 0x7d |
Third, I use the Python script provided by his Github repository.
from reedsolo import RSCodec
import reedsolo as rs
inputfile = open('input.bin', "rb")
n = 255
rsc = RSCodec(32)
rs.init_tables(0x11d)
ecc_encode = rsc.encode(inputfile.read())
If you print ecc_encode out, you can see the 32-byte RS code below.
| rs code | column 1 | column 2 | column 3 | column 4 | column 5 | column 6 | column 7 | column 8 | column 9 | column 10 | column 11 | column 12 | column 13 | column 14 | column 15 | column 16 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| row 1 | 0xfd | 0xe2 | 0xaa | 0x47 | 0x93 | 0xad | 0x57 | 0x5b | 0xca | 0xd8 | 0x17 | 0x19 | 0xe2 | 0x38 | 0x52 | 0x7d |
| row 2 | 0x74 | 0xa3 | 0x98 | 0xf9 | 0x23 | 0xc3 | 0xa3 | 0xba | 0x3d | 0x80 | 0x80 | 0x02 | 0x61 | 0xf5 | 0x91 | 0x29 |
The result generated by Python script is not corrected (at least, it is not my desired).
Could you help me correct the Python script? Thank you.
I'm sorry for being picky but the description right now says:
Pure-Python Reed Solomon encoder/decoder
However, the default install includes a C module, so that's not really "Pure-Python" and could lead to confusion. Would it be fine to change that to just "Python Reed Solomon encoder/decoder"?
an optimization for python 2.6 and above
Python 3.10.7
GCC 12.2.0
archlinux (manjaro)
Looks like a simple bug with the increment operator but could this somehow be due to a breaking change introduced by GCC 12 because I successfully installed it the other day on another machine with GCC 11.3 (I don't recall anything about this compilation showing but Cython is installed there too)
Is there any workaround with a binary or a way to ask pip to ignore the 'cython optimised module' ? I saw suggestions in #32 but they did not help.
command:
pip install reedsolo
output:
Using cached reedsolo-1.5.4.tar.gz (271 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: reedsolo
Building wheel for reedsolo (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [124 lines of output]
Cython is installed, building creedsolo module
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
copying reedsolo.py -> build/lib.linux-x86_64-cpython-310
running build_ext
building 'creedsolo' extension
creating build/temp.linux-x86_64-cpython-310
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug -flto=auto -fPIC -I/usr/include/python3.10 -c creedsolo.c -o build/temp.linux-x86_64-cpython-310/creedsolo.o
creedsolo.c: In function ‘__pyx_tp_dealloc_array’:
creedsolo.c:32027:5: error: lvalue required as increment operand
32027 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32029:5: error: lvalue required as decrement operand
32029 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc_memoryview’:
creedsolo.c:32338:5: error: lvalue required as increment operand
32338 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32340:5: error: lvalue required as decrement operand
32340 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
creedsolo.c:32588:5: error: lvalue required as increment operand
32588 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32590:5: error: lvalue required as decrement operand
32590 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__Pyx_InitGlobals’:
creedsolo.c:33787:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations]
33787 | PyEval_InitThreads();
| ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:130,
from creedsolo.c:16:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_ParseOptionalKeywords’:
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_decode_c_string’:
creedsolo.c:37955:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
37955 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for reedsolo
Running setup.py clean for reedsolo
Failed to build reedsolo
Installing collected packages: reedsolo
Running setup.py install for reedsolo ... error
error: subprocess-exited-with-error
× Running setup.py install for reedsolo did not run successfully.
│ exit code: 1
╰─> [126 lines of output]
Cython is installed, building creedsolo module
running install
/usr/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
copying reedsolo.py -> build/lib.linux-x86_64-cpython-310
running build_ext
building 'creedsolo' extension
creating build/temp.linux-x86_64-cpython-310
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug -flto=auto -fPIC -I/usr/include/python3.10 -c creedsolo.c -o build/temp.linux-x86_64-cpython-310/creedsolo.o
creedsolo.c: In function ‘__pyx_tp_dealloc_array’:
creedsolo.c:32027:5: error: lvalue required as increment operand
32027 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32029:5: error: lvalue required as decrement operand
32029 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc_memoryview’:
creedsolo.c:32338:5: error: lvalue required as increment operand
32338 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32340:5: error: lvalue required as decrement operand
32340 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
creedsolo.c:32588:5: error: lvalue required as increment operand
32588 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32590:5: error: lvalue required as decrement operand
32590 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__Pyx_InitGlobals’:
creedsolo.c:33787:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations]
33787 | PyEval_InitThreads();
| ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:130,
from creedsolo.c:16:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_ParseOptionalKeywords’:
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_decode_c_string’:
creedsolo.c:37955:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
37955 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> reedsolo
The following is mentioned in the README:
Although sanity checks are implemented whenever possible and when they are not too much resource consuming, there are a few cases where messages will not be decoded correctly without raising an exception:
- If an incorrect erasure location is provided, the decoding algorithm will just trust the provided locations and create a syndrome that will be wrong, resulting in an incorrect decoded message. In case reliability is critical, always use the check() method after decoding to check the decoding did not go wrong.
A fairly straightforward method to enable variable rate coding would be to not transmit/store trailing error or data symbols, so you could increase or decrease the rate at will, respectively. Half of this functionality is already fully available, since if you skip error symbols, you can simply mark them as having been erased.
However, if you choose to skip data symbols (input them as zero and then don't store/transmit), there is no way to inform the decoder that these zeroed symbols (which will have been substituted back in as zeros before calling the decoder) are definitely correct.
For this and perhaps other uses that I am not aware of yet, a feature by which you could do the opposite of the existing functionality — inform the decoder that certain symbols are definitely correct — could be extremely useful.
here's the relevant snippets of my code, which works fine with errors not exceeding RS capability
N, K = 255, 223
rsc = RSCodec(N-K, nsize=N)
for i in range(0,Npixels*N//K,N):
try:
rx_byte = np.append(rx_byte,np.uint8(rsc.decode(rx_enc[i:i+N])[0]))
except ReedSolomonError:
rx_byte = np.append(rx_byte,rx_enc[i:i+K])I get
Traceback (most recent call last):
File " ", line 36, in <module>
rx_byte = np.append(rx_byte,np.uint8(rsc.decode((rx_enc[i:i+N]))[0]))
File "C:\ProgramData\Anaconda3\lib\site-packages\reedsolo.py", line 924, in decode
rmes, recc, errata_pos = rs_correct_msg(chunk, nsym, fcr=self.fcr, generator=self.generator, erase_pos=e_pos, only_erasures=only_erasures)
File "C:\ProgramData\Anaconda3\lib\site-packages\reedsolo.py", line 747, in rs_correct_msg
err_pos = rs_find_errors(err_loc[::-1], len(msg_out), generator)
File "C:\ProgramData\Anaconda3\lib\site-packages\reedsolo.py", line 693, in rs_find_errors
raise ReedSolomonError("Too many (or few) errors found by Chien Search for the errata locator polynomial!")
ReedSolomonError: Too many (or few) errors found by Chien Search for the errata locator polynomial!
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File " ", line 37, in <module>
except ReedSolomonError:
NameError: name 'ReedSolomonError' is not definedpip install reedsolo
fails with:
Collecting reedsolo
Using cached reedsolo-1.5.4.tar.gz (271 kB)
Building wheels for collected packages: reedsolo
Building wheel for reedsolo (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\WinPython\WPy64-3860\python-3.8.6.amd64\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\xxxxx\\AppData\\Local\\Temp\\pip-install-sknnm65z\\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\\setup.py'"'"'; __file__='"'"'C:\\Users\\xxxxx\\AppData\\Local\\Temp\\pip-install-sknnm65z\\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\xxxxx\AppData\Local\Temp\pip-wheel-_7lq_i9f'
cwd: C:\Users\xxxxx\AppData\Local\Temp\pip-install-sknnm65z\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\
Complete output (10 lines):
Cython is installed, building creedsolo module
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.8
copying reedsolo.py -> build\lib.win-amd64-3.8
running build_ext
building 'creedsolo' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
----------------------------------------
ERROR: Failed building wheel for reedsolo
Running setup.py clean for reedsolo
Failed to build reedsolo
Installing collected packages: reedsolo
Running setup.py install for reedsolo ... error
ERROR: Command errored out with exit status 1:
command: 'C:\WinPython\WPy64-3860\python-3.8.6.amd64\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\xxxxx\\AppData\\Local\\Temp\\pip-install-sknnm65z\\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\\setup.py'"'"'; __file__='"'"'C:\\Users\\xxxxx\\AppData\\Local\\Temp\\pip-install-sknnm65z\\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\xxxxx\AppData\Local\Temp\pip-record-qw0ikq9c\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\WinPython\WPy64-3860\python-3.8.6.amd64\Include\reedsolo'
cwd: C:\Users\xxxxx\AppData\Local\Temp\pip-install-sknnm65z\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\
Complete output (10 lines):
Cython is installed, building creedsolo module
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.8
copying reedsolo.py -> build\lib.win-amd64-3.8
running build_ext
building 'creedsolo' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\WinPython\WPy64-3860\python-3.8.6.amd64\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\xxxxx\\AppData\\Local\\Temp\\pip-install-sknnm65z\\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\\setup.py'"'"'; __file__='"'"'C:\\Users\\xxxxx\\AppData\\Local\\Temp\\pip-install-sknnm65z\\reedsolo_d0f8e6cce4fe4abfbb92aa3abc1928b7\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\xxxxx\AppData\Local\Temp\pip-record-qw0ikq9c\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\WinPython\WPy64-3860\python-3.8.6.amd64\Include\reedsolo' Check the logs for full command output.
pip installation should run without need to install massive ms toolchain. Provide wheels?
I'm trying this library with error correcting on network sent messages (between two VMs), but something is wrong. When network connection is perfect and there's no packet loss encoding and decoding works perfectly, but when I'm simulating packet loss even with 4 or 5 errors I get error message:
File "/usr/local/lib/python3.8/dist-packages/reedsolo.py", line 924, in decode
rmes, recc, errata_pos = rs_correct_msg(chunk, nsym, fcr=self.fcr, generator=self.generator, erase_pos=e_pos, only_erasures=only_erasures)
File "/usr/local/lib/python3.8/dist-packages/reedsolo.py", line 745, in rs_correct_msg
err_loc = rs_find_error_locator(fsynd, nsym, erase_count=len(erase_pos))
File "/usr/local/lib/python3.8/dist-packages/reedsolo.py", line 658, in rs_find_error_locator
raise ReedSolomonError("Too many errors to correct")
I'm working with ECC=13 and I even track errors and their positions which I enter in decoding and still.. Any ideas?
EDIT:
Got this:
Message sent: bytearray(b'123456rx\x15\xa5\x81\x88\xab\xd5\x0c\xf1E\xbc\xce')
Message received:
[14]
bytearray(b'123456'), bytearray(b'123456rx\x15\xa5\x81\x88\xab\xd5\x0c\xf1E\xbc\xce'), bytearray(b'\x12\x11\x10\x0f\x0e\r')
[14] - is error position by my calculation (14th byte)
EDIT No. 2:
My bad, sorry :) Kinda works, though, error positions that my code shows and reedsolo show still aren't identical
Mirror of issue lrq3000#3
In the file reedsolo.py at line 499: magnitude = gf_div(y, err_loc_prime)
this will divide by zero if err_loc_prime is zero.
You should add a check for err_loc_prime == 0
like this:
y = gf_poly_eval(err_eval[::-1], Xi_inv) # numerator of the Forney algorithm (errata evaluator evaluated)
y = gf_mul(gf_pow(Xi, 1), y)
if err_loc_prime == 0:
raise ReedSolomonError("Could not find error magnitude")
# Compute the magnitude
magnitude = gf_div(y, err_loc_prime) # magnitude value of the error, calculated by the Forney algorithm (an equation in fact): dividing the errata evaluator with the errata locator derivative gives us the errata magnitude (ie, value to repair) the ith symbol
E[err_pos[i]] = magnitude # store the magnitude for this error into the magnitude polynomial
I'm using the following code:
import creedsolo as rs
prim = rs.find_prime_polys(c_exp=12, fast_primes=False, single=True)
rs.init_tables(prim=prim, c_exp=12)The error thrown is:
c_gf_log, c_gf_exp, c_field_charac = rs.init_tables(prim=prim, c_exp=12)
File "creedsolo.pyx", line 203, in creedsolo.init_tables
OverflowError: value too large to convert to unsigned char
I'm trying to encode a file with 4095 sized chunks without using the additional 255 internal splittings. I've precalculated the polynomial numbers. So far I was able to encode, but not to decode the message:
import creedsolo as rs
rs.init_tables()
mesecc = rs.rs_encode_msg(data, nsym, gen=gen[nsym]) # nsym = 123, len(data) = 3972
rmes, rmesecc, errata_pos = rs.rs_correct_msg(mesecc, nsym)The error thrown is:
rmes, rmesecc, errata_pos = rs.rs_correct_msg(mesecc, nsym)
File "creedsolo.pyx", line 694, in creedsolo.rs_correct_msg
ValueError: Message is too long (4095 when max is 255)
Is there a way to encode 4095 chunks without hidden splitting?
Data length = 64 bit (data) + 32 bit (RS padding) = 96 bit.
Current code is trying to correct in Byte Unit.
But I want to correct in bit unit so that it can correct as many bit as possible.
Thank you!
Is it possible to use RSCodec as a binary codec? If so, how do I do that? I've tried limiting the symbol alphabet by setting c_exp to 2 (edit: *set c_exp to 1) only to be greeted with this error message:
File "<stdin>", line 1, in <module> File "/opt/homebrew/lib/python3.9/site-packages/reedsolo.py", line 869, in __init__ self.gen[nsym] = rs_generator_poly(nsym, fcr=fcr, generator=generator) File "/opt/homebrew/lib/python3.9/site-packages/reedsolo.py", line 484, in rs_generator_poly g = gf_poly_mul(g, [1, gf_pow(generator, i+fcr)]) File "/opt/homebrew/lib/python3.9/site-packages/reedsolo.py", line 331, in gf_pow return gf_exp[(gf_log[x] * power) % field_charac] IndexError: bytearray index out of range
I think someone has attempted to do the same thing here.
Python: 3.10
Reedsolo: 1.5.4
OS: Linux / Slackware
Collecting reedsolo
Using cached reedsolo-1.5.4.tar.gz (271 kB)
Building wheels for collected packages: reedsolo
Building wheel for reedsolo (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yfvlvd6u/reedsolo_8cc3b5cd27ba467ea55234248fb4edf0/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yfvlvd6u/reedsolo_8cc3b5cd27ba467ea55234248fb4edf0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-0hk94tmt
cwd: /tmp/pip-install-yfvlvd6u/reedsolo_8cc3b5cd27ba467ea55234248fb4edf0/
Complete output (161 lines):
Cython is installed, building creedsolo module
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-i686-3.10
copying reedsolo.py -> build/lib.linux-i686-3.10
running build_ext
building 'creedsolo' extension
creating build/temp.linux-i686-3.10
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include/python3.10 -c creedsolo.c -o build/temp.linux-i686-3.10/creedsolo.o
creedsolo.c: In function ‘__pyx_tp_dealloc_array’:
creedsolo.c:32027:5: error: lvalue required as increment operand
32027 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32029:5: error: lvalue required as decrement operand
32029 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc_memoryview’:
creedsolo.c:32338:5: error: lvalue required as increment operand
32338 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32340:5: error: lvalue required as decrement operand
32340 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
creedsolo.c:32588:5: error: lvalue required as increment operand
32588 | ++Py_REFCNT(o);
| ^~
creedsolo.c:32590:5: error: lvalue required as decrement operand
32590 | --Py_REFCNT(o);
| ^~
creedsolo.c: In function ‘__Pyx_InitGlobals’:
creedsolo.c:33787:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations]
33787 | PyEval_InitThreads();
| ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:144,
from creedsolo.c:16:
/usr/include/python3.10/ceval.h:122:37: note: declared here
122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
| ^~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_ParseOptionalKeywords’:
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35153:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35153 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
creedsolo.c:35169:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
35169 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
creedsolo.c: In function ‘__Pyx_decode_c_string’:
creedsolo.c:37955:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
37955 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:96,
from creedsolo.c:16:
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for reedsolo
Thanks!
Example failure
import reedsolo
rsc = reedsolo.RSCodec(30)
encoded = rsc.encode(b'0' * 226)
_, _, _ = rsc.decode(encoded, erase_pos=[255], only_erasures=True)
output:
Traceback (most recent call last):
File "demo.py", line 4, in <module>
_, _, _ = rsc.decode(encoded, erase_pos=[255], only_erasures=True)
File "reedsolo.py", line 927, in decode
rmes, recc, errata_pos = rs_correct_msg(chunk, nsym, fcr=self.fcr, generator=self.generator, erase_pos=e_pos, only_erasures=only_erasures)
File "reedsolo.py", line 731, in rs_correct_msg
msg_out[e_pos] = 0
IndexError: bytearray index out of range
Erasure position calculation for the 2nd chunk onward are incorrect due to this line:
922 e_pos = [x for x in erase_pos if x <= self.nsize]
923 ...
924 erase_pos = [x - (self.nsize+1) for x in erase_pos if x > self.nsize]
it should be:
922 e_pos = [x for x in erase_pos if x < self.nsize]
923 ...
924 erase_pos = [x - (self.nsize) for x in erase_pos if x >= self.nsize]
Hi,
thank you for the library.
I am not sure if it is possible but I am currently trying to encode a single value of 20 bits, so I am trying to feed the encoder with a boolean array of length 20. e.g. [true, true, false, true, false ....]
After encoding, 60 bits should be produced (in total)
I would like to be able to correct 5 errors.
Size of the alphabet q = 16
blocklength n = 15
Message length k = 5
I am not sure what parameters I have to use - of if I am using them correctly.
I have a legacy Java implementation, but the produced data differs....
Can somebody help me to understand the correct use of the library for the parameters above...?
Hi! I'm packaging this module for NixOS. It seems that the pypi tarball does not include the tests dir, so I'm using the source from the github repo. It would be useful if the latest release, v1.5.4 were tagged here.
git tag v1.5.4 73926cdf81b39009bd6e46c8d49f3bbc0eaad4e4
git push origin v1.5.4
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.