nicoretti / crc Goto Github PK
View Code? Open in Web Editor NEWPure Python CRC library
Home Page: https://nicoretti.github.io/crc/
License: BSD 2-Clause "Simplified" License
Pure Python CRC library
Home Page: https://nicoretti.github.io/crc/
License: BSD 2-Clause "Simplified" License
Similar to the previous automated reporting using Travis CI
Either continue using:
Something similar like the deployment once in place with Travis CI should be possible
Create a script which takes care of release a new version
To start with, should this library be usable with CRC algorithms that operate on bits instead of bytes (eg. nor data nor result is byte-sized) ?
If so, i'd appreciate help with configuration for CRC-9 per ETSI DMR specification
Page 146, section B.3.10, https://www.etsi.org/deliver/etsi_ts/102300_102399/10236101/02.05.01_60/ts_10236101v020501p.pdf
Polynomial in normal form is 0x59 (bits 9,6,4,3,0 set is 0x259, removing MSB bit then 0x59)
From formula, the CRC result should be inverted, however same document (page 144, section B.3.8 CRC-16-CCITT) shows the inversion polynomial for CRC-16-CCITT, and in your config of the same no inversion is configured ( https://github.com/Nicoretti/crc/blob/master/crc.py#L387 )
I've tried to get it working by using this config, which produces incorrect result
CALC: CrcCalculator = CrcCalculator(
configuration=Configuration(
width=9,
polynomial=0x59,
init_value=0x00,
reverse_input=False,
reverse_output=True,
final_xor_value=0x00,
),
table_based=True,
)
Testing data are for example
bitarray('000000010000000000000000000000010000000100000000010011010000000001001101000000000100011100000000010101000000000000101110000000000000000')
This is 135 bits of data, resulting CRC9 should be either 409 or 102
Currently, input files are read in text mode, converting using the default selected encoding (system dependent) to python 'char's, then converting to bytearray using the 'utf8' encoding, which may or may not be the same encoding as how the file was read originally.
This could lead to surprises if the read() and bytes() encodings are different.
It might be better (and surely more efficient) to open files with "rb" (line 494: ...FileType('rb')) and use the sys.stdin.buffer as default, so that the read() method returns bytearray directly (like it used to do in python2).
I tried to run the example Create a custom crc configuration for the crc calculation
in the readme.
How ever It showed following error on Python 3.8.10:
Traceback (most recent call last):
File "crctest.py", line 15, in <module>
crc_calculator = CrcCalculator(configuration, use_table)
File "/usr/local/lib/python3.8/dist-packages/crc.py", line 332, in __init__
self._crc_register = TableBasedCrcRegister(configuration)
File "/usr/local/lib/python3.8/dist-packages/crc.py", line 230, in __init__
super().__init__(configuration)
File "/usr/local/lib/python3.8/dist-packages/crc.py", line 104, in __init__
self._register = configuration.init_value & self._bitmask
TypeError: unsupported operand type(s) for &: 'tuple' and 'int'
The solution was to changed all configs from tuples to normal values by removing all appended commas:
width = 8
poly=0x07
init_value=0x00
final_xor_value=0x00
reverse_input=False
reverse_output=False
The check failed afterwards, however it works correctly for the different Poly (0xE7) .
Tested with following patterns:
data = [0x12]
data = [0xFF]
data = [0x34, 0x12]
data = [0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc, 0xde]
All lookup table entries starting with '0' nibbles are output wrongly in the 'table' mode. For instance crc16 CCITT index 68 should be 0x0840, but it comes out 0x8400.
The predefined Crc16.CCITT
has a misleading name. Referring to CRC-16/KERMIT in the CRC catalog, which says it has alias CRC-CCITT, I would expect it to implement that. But it instead appears to implement CRC-16/XMODEM.
I would expect the predefined Crc16.CCITT
to be equivalent to
crc_ccitt_config = crc.Configuration(width=16, polynomial=0x1021, reverse_input=True, reverse_output=True)
And so calculating the "check" value:
crc_ccitt = crc.Calculator(crc_ccitt_config)
hex(crc_ccitt.checksum(b'123456789'))
'0x2189'
But, that's not the definition of Crc16.CCITT
. Rather, it appears to really be CRC-16/XMODEM:
crc_ccitt_really_xmodem_config = crc.Crc16.CCITT
crc_ccitt_really_xmodem = crc.Calculator(crc_ccitt_really_xmodem_config)
hex(crc_ccitt_really_xmodem.checksum(b'123456789'))
'0x31c3'
It would be helpful if the documentation has a table of all the predefined CRC algorithms, listing all their configuration parameters, along with a link to the CRC catalog..
I use the module with the following configuration.
crc.CrcCalculator(
crc.Configuration(
width=16,
polynomial=0x8005,
init_value=0x0000,
final_xor_value=0,
reverse_input=True,
reverse_output=False
),
table_based=True
)
Performance when crunching big payloads was in the range that I expected.
But while profiling other code I noticed that a vast amount of time was spent simply reversing the input bytes.
I now use the module without reversed_input and reverse the input externally (bytes.translate) which I propose should be done inside the library when running in Table Based Mode, as the memory overhead is negligible, but leads to a 45% speedup in my setup.
Alternatively I think one could transpose the whole lookup table when working with reversed input to get even better performance.
Greetings to the maintainers of this very helpful package.
Line 95 in 980d08c
"poly" should be "polynomial" in example
currently read the docs linke points to an outdated version of the documentation.
Could you please add the following CRC8 definition for MAXIM 1-wire devices (like DS18B20) to the Crc8 enum?
MAXIM = Configuration(
width=8,
polynomial=0x131,
init_value=0,
final_xor_value=0,
reverse_input=True,
reverse_output=True,
)
make sure if one clicks on the badge he/she is redirected to the actual service instead of the badge itself
In your example you are using a method that does not exist.
assert crc_calculator.verfify_checksum(data, expected_checksum)
Using GNURADIO I saved AIS vessel information in file Radio reception. Using Python code I look for the preamble and start signal. so after bit destuffing, I try to check the FCS with a crc16 calculation.
The data from the AIS recorded (HDLC)
'0x124174d2160000f5791afa2c302987f07fa57b0060'
# or
'0x120x410x740xd20x160x000x000xf50x790x1a0xfa0x2c0x300x290x870xf00x7f0xa50x7b0x000x60'
# or
'0x12 0x41 0x74 0xd2 0x16 0x00 0x00 0xf5 0x79 0x1a 0xfa 0x2c 0x30 0x29 0x87 0xf0 0x7f 0xa5 0x7b 0x00 0x60'
The CRC(from FCS part of HDLC)
'0x1b18'
those data works fine under this application : http://www.sunshine2k.de/coding/javascript/crc/crc_js.html
note : The application uses this format : 0x12 0x41 0x74 0xd2 0x16 0x00 0x00 0xf5 0x79 0x1a 0xfa 0x2c 0x30 0x29 0x87 0xf0 0x7f 0xa5 0x7b 0x00 0x60
MAYBE
in your example you used bytes([1,2,3,4,5]) --> those bytes function return b'\x01\x02\x03\x04\x05'
but if you use value above 9 you have strange characters (as if ASCII transcription was done).
for example if I use :
'124174d2160000f5791afa2c302987f07fa57b0060'
And a byte.fromhex('124174d2160000f5791afa2c302987f07fa57b0060')
b'\x12At\xd2\x16\x00\x00\xf5y\x1a\xfa,0)\x87\xf0\x7f\xa5{\x00`'
the character are strange and of course the value from verify is False..
CONFIGURATION
SYSTEM :
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
Python : 3.11.4
Thank you for your help
Make sure if one clicks on a badge it will redirect to the build, etc instead of redirecting to the badge image itself.
diff --git a/crc.py b/crc.py
index 3a92bde..ad728fa 100644
--- a/crc.py
+++ b/crc.py
@@ -492,6 +492,15 @@ class Crc16(enum.Enum):
reverse_output=False,
)
+ MODBUS = Configuration(
+ width=16,
+ polynomial=0x8005,
+ init_value=0xFFFF,
+ final_xor_value=0x0000,
+ reverse_input=True,
+ reverse_output=True,
+ )
+
Multiple calls to digest()
don't return the same answers.
>>> import crc
>>> r = crc.Register(crc.Crc8.BLUETOOTH)
>>> r.init()
>>> r.update(b"Hello World!")
138
>>> r.digest()
81
>>> r.digest()
138
>>> r.digest()
81
Expected multiple calls to digest()
to return the same value.
Return value flip-flops between two values.
I think it is caused by in-place reverse in digest()
, L256-L257
First of all, thank you for this great package! I use it extensively and it works more than fine for me...
As a minor inconvenience, I have seen that there are no __version__
attribute existing anymore, which bothered me while I was trying to perform some test on it to improve the portability of some of my code to older systems. Solved this with a try
/catch
, but it is not ideal at all. I think it would be good practice to have a __version__
value against which to test, especially when some features/functions are not back-compatible :-)
While the exact same configuration yield a valid crc checksum when using the primitives of the library directly,
using the Calculator
class yields a wrong result.
Reproducibility: sporadic, regularly, always
from crc import Configuration, Calculator
config = Configuration(
width=16,
polynomial =0x1021,
init_value=0xffff,
final_xor_value=0xffff,
reverse_input=False,
reverse_output=False,
)
calc = Calculator(config)
expected = 0x1B18
data = [18, 65, 116, 210, 22, 0, 0, 245, 121, 26, 250, 44, 48, 41, 135, 240, 127, 165, 123, 0, 96]
assert expected == calculator.checksum(data)
The resulting checksum is 0x1B18
The resulting checksum is 0x356c
?
The following snippet generates the correct checksum:
from crc import Configuration, Register
config = Configuration(
width=16,
polynomial =0x1021,
init_value=0xffff,
final_xor_value=0xffff,
reverse_input=False,
reverse_output=False,
)
reg = Register(config)
expected = 0x1B18
data = [18, 65, 116, 210, 22, 0, 0, 245, 121, 26, 250, 44, 48, 41, 135, 240, 127, 165, 123, 0, 96]
reg.init()
reg.update(data)
assert expected == reg.digest()
Recently issues regarding the cli tools provided by the crc package have been/are addressed.
Once all of them are merged create a pypi release containing all those improvments.
Travis CI seems to have moved, either change to use their new service and service address (update tokens etc.) or switch to Github Actions.
What needs to be setup
My pytest tests that used to work 3 days ago failed today on github.
While it is possible it is my bug, it seems to be related to crc CCITT.
Reproducibility: always
it is able to run tests
it is not able to run tests, but it was 3 days ago
I can run tests ok locally on windows when I install
crc==4.2.0
Error
<string>:13: in __init__
???
src/ncu_backend/ncu_model/data_layer/nbit_status.py:73: in __post_init__
self.update_crc()
src/ncu_backend/ncu_model/data_layer/nbit_status.py:213: in update_crc
self._crc = self._calc_crc()
src/ncu_backend/ncu_model/data_layer/nbit_status.py:226: in _calc_crc
Crc16.CCITT
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/enum.py:437: in __getattr__
raise AttributeError(name) from None
E AttributeError: CCITT
Github running pytest on python 3.10
Desktop:
use build in cli parser, so no external dependencies are required
The "Available CRC Configurations" section of the readme is to be updated, with respect to the attribute "SAJ1850"
Under the "Available CRC Configurations" section of the readme:
Under CRC8, there is an attribute named "SAJ1850". However, using this attribute for CRC calculation causes an AttributeError: SAJ1850
. Using the attribute "SAEJ1850" works, however. So the documentation can be updated to mention "SAEJ1850" instead of "SAJ1850".
The section also states "For convince various frequently used crc configurations ship with the library out of the box." - is it meant to state "For convenience" instead? If yes, this typo can be corrected.
Why should this documentation be added/updated?
Using the attribute "SAJ1850" instead of "SAEJ1850" for CRC calculation causes an AttributeError: SAJ1850
.
Possible typo in the documentation: "For convince various frequently used crc configurations ship with the library out of the box." - is it meant to state "For convenience" instead?
https://nicoretti.github.io/crc
Thanks!
Thanks for having type annotation in your package! However, I can't use it with type checkers in my own code, because crc
is missing a py.typed
marker file. Please see PEP 561 for more information.
This leads to errors in downstream projects using type checkers like mypy:
error: Skipping analyzing "crc": module is installed, but missing library stubs or py.typed marker [import]
note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Looking at your source code package...:
$ tar -tf crc-4.3.0.tar.gz
crc-4.3.0/LICENSE.txt
crc-4.3.0/README.md
crc-4.3.0/crc.py
crc-4.3.0/pyproject.toml
crc-4.3.0/setup.py
crc-4.3.0/PKG-INFO
... I understand your current distribution is single-file/module-only, but it seems that PEP requires a refactor into a package structure.
This PEP does not support distributing typing information as part of module-only distributions or single-file modules within namespace packages.
The single-file module should be refactored into a package and indicate that the package supports typing as described above.
Let me know if you need some help with that; I'm happy to provide a PR.
The current Crc16.CCITT
implementation in our library aligns with CRC-16/XMODEM
, which isn't listed among the alternative names/"Aliases" for "CCITT". One of the common names this configuration is known by is KERMIT (see here). We need to ensure CCITT provides the correct configuration or is replaced by at least one of the valid known alternative names/aliases.
Crc16.CCITT
to Crc16.XMODEM
to accurately represent its configuration.Crc16.CCITT
with the appropriate configuration.Crc16.KERMIT
(optional).Expected Outcome:
The library will accurately represent various CRC-16 configurations, reducing confusion and increasing usability.
amount_of_bits(input) % 8 == 0
must hold trueCrc8.SAEJ1850
should have init_value=0xff
and final_xor_value=0xff
.
SAE J1850-2001 says "7.4.1 [...] a. The CRC calculation [...] shall be initially set to the 'all ones' state [...]" and "e. The CRC byte is made equal to [...] the ones complement of R(X)," i.e. XOR with 0xFF
.
This also matches the configuration used in crc-rs.
I am currently using the latest release of this module crc-7.0.0
.
The result is wrong when the input data is a list of int:
>>> from crc import Calculator, Crc32
>>>
>>> d1 = 35
>>> d2 = [35]
>>> d3 = bytes([35])
>>>
>>> calc = Calculator(Crc32.CRC32)
>>>
>>> calc.checksum(d1)
1885708031
>>> calc.checksum(d2)
167270199
>>> calc.checksum(d3)
1885708031
>>>
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.