python-intelhex / intelhex Goto Github PK
View Code? Open in Web Editor NEWPython IntelHex library
License: BSD 3-Clause "New" or "Revised" License
Python IntelHex library
License: BSD 3-Clause "New" or "Revised" License
Running the tests gives me
======================================================================
FAIL: test_xrange_longint (intelhex.test.TestXrangeLongInt)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/nix-build-python2.7-intelhex-2.1.drv-0/intelhex-2.1/intelhex/test.py", line 1601, in test_xrange_longint
self.assertRaises(OverflowError, xrange, 2684625744, 2684625747)
AssertionError: OverflowError not raised
----------------------------------------------------------------------
trying something similar in a python interpreter gives the same results:
$ python
Python 2.7.13 (default, Dec 17 2016, 20:05:07)
[GCC 5.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> xrange(2684625744, 2684625747)
xrange(2684625744, 2684625747)
while something like this does in fact throw an OverflowError:
>>> xrange(2684625744, 2**100)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long
Hi,
I am looking for effective way of hex file data comparison, mainly equality check.
Is there already some effective way of comparison?
Did you consider overriding of __eq__
?
Hello,
my issue might be considered more as improvement. So I don't know if this is the right place.
I've observed that when You want to use write_hex_file, then You cannot decide which type of Extend Address record type will be used in hex file:
Always second one will be used (0x04) and it cannot be changed as far I can see.
The code responsible
while cur_addr <= maxaddr:
if need_offset_record:
bin = array('B', asbytes('\0'*7))
bin[0] = 2 # reclen
bin[1] = 0 # offset msb
bin[2] = 0 # offset lsb
bin[3] = 4 # rectyp
I know that having that record type (0x04) shouldn't be a problem for most cases, but I have to deal with some legacy software. I run into problems when this record is not 0x02.
Is there any chance to add an option for 'write_to_hex' to have possibility to have functionality tp force using Extended Segment Address record types in output file.
In particular scenario when You initialize IntelHex with file that contains this kind of records, writing this with 'write_hex_file' (without changing any byte of payload) will result with differences comparing to original.
I can try to create a pull request with changes by myself, but I'm not very experienced Python programmer. I do Embedded (mostly C), but I starting to use Python more and more, especially for tooling.
Altough it is not normally supported by the .hex format, many vendors embed comments in their .hex file, which make them impossible to parse by IntelHex unless pre-cleaned programmatically.
It would be nice to be able to specify an optional comment character (;, #, --) that would make the parser ignore any line starting with it.
Or maybe ignore everything not starting with a record delimiter.
From pull request #7 (comment)
Example of hex file:
:01000000bd42
:00000001FF
On windows, both hex2dump and hex2bin work.
On Linux, only hex2dump (the character is not display correctly but I don't
care)
Python versions: On windows python 3.4.3 and on linux 3.4.1.
https://bugs.launchpad.net/intelhex/+bug/372625
Old bug from LP. Moving it here.
The most important bits:
Standard range function has 3 parameters: start, stop [,step]
here: stop is exclusive
My tobinXXX group of methods first parameters are: start, end
here: end is inclusive
I have 2 choices:
Require API break or semi-break.
I saw in loadhex() is supported the record type == 2 (IHEX_EXTENDED_SEGMENT_ADDRESS_RECORD), but in write_hex_file(), there is not supported this record type?
Thanks.
Line 60 in 9769d3f
tostring has to become tobytes for this to work in Python 3.9, likely in versions before this as well.
When I use this package to generate memory initializing file for alera_mf MEM 1 port with 16 bit word length, the saved hex file is working. I create and use mif file format. It is working. When I convert mif file to hex file using Quartus II (13.1), the different are: 1. byte_count on each line is 2 and 2. the address are word address not byte address. Although I can add byte_count=2 when writing to hex file, the address there is no way to change the addresses.
ih.write_hex_file(f_hex, byte_count=2)
Use intelhex package:
:0200000080007E
:02000200800676
:02000400800C6E
Convert using Quartus II
:0200000080007E
:02000100800677
:02000200800C70
To solve the problem, I think creating a new class support any word length, and option for word address when writing hex file.
Consider the following.
ih = IntelHex({0:0, 1:1, 2:2})
# The bug
print(ih[0:0].tobinarray()) # array('B', [0, 1, 2])
# Correct for other indices
print(ih[1:1].tobinarray()) # array('B', [])
# stdlib __getitem__ for reference
li = [1, 2, 3]
print(li[0:0]) # []
Looking at the source for _getitem_()
this behavior happens because the endaddr 0 evaluates the same as None.
The fix should simply be:
# Current
start = addr.start or addresses[0]
stop = addr.stop or (addresses[-1]+1)
# Fixed
start = addr.start if addr.start is not None else addresses[0]
stop = addr.stop if addr.stop is not None else (addresses[-1]+1)
Hi, I help support the PyPI server software by answering bug reports. We've recently been told that some of the older releases of this project (1.5 and below, I think) have a "Home Page" URL specified that points to a compromised server that is deploying malware.
Could you please check that out? If true, since it's not possible to edit older releases anymore, then if the server cannot be fixed, the only option may be to remove those older releases.
Thank you.
python 9 is not supporting the tostring() attribute of array
from intelhex import IntelHex
ih = IntelHex('pimp_my_xor.hex')
ih.minaddr()
256
ihx = ih[0x100:0x103]
ihx.tobinstr()
b'\x89\xc2\x8d'
list(ihx)
[255, 255, 255, 255, 255, 255, 255, ..., 255, 255, 137, 194, 141]
I am testing this module for some production/testing firmware automation and I am super pleased!
I am noticing that when I encode a hex file to a dict with this module, then fetch and re-save to a hex file (the data is stored in a database as a JSON record) one line gets swapped in the file thus causing my md5 checksums to not match. While I am fairly certain the structure of an intel hex file prevents this from being an issue, it would be nice to under stand how this happens.
Encode to database:
file_name = file_path.split('/')[-1]
with open(file_path, 'rb') as file:
md5 = hashlib.md5(
file.read()
).hexdigest()
intel_hex = IntelHex()
intel_hex.loadhex(file_path)
if image_type.lower() in ['mfg', 'manufacturing']:
self.manufacturing_fw = intel_hex.todict()
self.manufacturing_fw_name = file_name
self.manufacturing_md5 = md5
Decode From database:
if image_type.lower() in ['mfg', 'manufacturing']:
output_file_name = f'{out_path}/{self.part_number}-{self.manufacturing_fw_name}'
firmware_data = self.manufacturing_fw
data = {}
for key, value in firmware_data.items():
if key != 'start_addr':
key = int(key)
data[key] = value
intel_hex = IntelHex()
intel_hex.fromdict(data)
intel_hex.write_hex_file(output_file_name)
I need to add these lines upon export to re-arrange the lines to get the exported file to match the original exactly.
Not efficient or the minimum solution, but it works with everything I have tested so far.
file = open(output_file_name, 'r')
lines = file.readlines()
file.close()
output = open(output_file_name, 'w')
total_lines = len(lines)
last_line = lines[-1]
lines.append(lines[-1])
lines[0], lines[total_lines-1] = lines[total_lines-1], lines[0]
lines.append(last_line)
for line in lines[1:-1]:
output.write(line)
output.close()
I appreciate any help with this!
Hello,
Your library has been very helpful for reading .hex files.
Context: I'm using the IntelHex library to read from a .hex file, which I then use to flash a microcontroller.
As you can appreciate, there are thousands of bytes contained in the .hex, and so I'm attempting to read and send the maximum number of bytes in one message at a time, which is 4095.
My code looks something like this:
startAddress = 0x00840000
maxFrameSize = 4095
data = []
""" Send 100 blocks of data from hex file """
for blocks in range(100):
""" Increment sequence number for every data block sent """
for sequenceNumber in range(0x01, 0xFF):
""" Read data from .hex and append to array """
for i in range(maxFrameSize):
data.append(int(IH[startAddress + I]))
""" Send """
client.transfer_data(sequence_number= sequenceNumber, data= bytes(data))
""" Clear array """
data.clear()
""" Update Start address for next block """
startAddress += maxFrameSize
The issue I've encountered is that some bytes are swapped according to my trace, preventing me from finishing the data transfer.
The image is a diff containing my trace on the left, and a trace generated from another vendor-supplied microcontroller flashing tool, with the differences highlighted.
I hope I've provided enough information.
Any help would be appreciated!
I'd like to have some extras for pip install, e.g. optional tests, benchmark and scripts.
>>> sys.version_info
sys.version_info(major=3, minor=5, micro=3, releaselevel='final', serial=0)
>>> import intelhex
>>> myhex = intelhex.IntelHex('/home/albert/albert.kurucz/MultiRadio/trunk RF200_AES128_SnapV2.8.2.hex')
>>> myhex.segments()
[(0, 420), (752, 761), (3072, 3360), (65536, 122354), (128256, 128562), (128682, 128690), (128698, 128934)]
>>> data=myhex.gets(0, 420)
>>> type(data)
<class 'str'>
with python3 expected data type returned by gets is <class 'bytes'> not <class 'str'>
intelhex==2.1
At the moment the only possible data field byte count is 16 when using write_hex_file.
Intelhex wiki: The maximum byte count is 255 (0xFF). 16 (0x10) and 32 (0x20) are commonly used byte counts.
Should be able to choose variable byte_count ranging from 1 to 255 when writing the hex file.
There are some changes waiting to be released, and some pull requests waiting for me as well.
On the readthedocs site, I can click on hex2bin in the scripts category, but searching for the term does nothing. On either Firefox or Chrome-based Edge browsers, I get "Searching..." with blinking dots, but it never goes anywhere.
According to: https://en.wikibooks.org/wiki/X86_Assembly/16,_32,_and_64_Bits
... to create a 20 bit address, it would be done by taking the 16-bit value of a segment register and put it on the address bus,
but shifted 4 times to the left (thus effectively multiplying the register by 16), and then by adding the offset from another
register untouched to the value on the bus, thus creating a full a 20-bit address.
If CS = 258C and IP = 001216, then CS:IP will point to a 20 bit address equivalent to "CS ร 16 + IP" which will be
258C ร 1016 + 001216 = 258C0 + 001216 = 258D2 (Remember: 16 decimal = 1016).
However the implementation is: https://github.com/python-intelhex/intelhex/blob/master/intelhex/scripts/hexinfo.py
if keys == ['CS','IP']:
entry = ih.start_addr['CS'] * 65536 + ih.start_addr['IP']
Is it not an implementation mistake?
from intelhex import IntelHex
ih = IntelHex()
ih[0] = 0x55
list(ih)
Any binary file over 4K will only be transforms the first 4Kbytes. The rest are discard.
If I use the lib to generate hex file on the OS Win everything is ok
If I use the lib to generate hex file on the Unix OS and after this I try to upload this file to chip on Win OS a I will get a error.
Problem in the end of line symbol. Because Unix use only LF, but windows software expect a LFCR.
In python, using the in
operator is used to test for membership, thus it would be natural to expect that the following expression is valid to test if the given address is present in the hex. However it hangs of an unknown reason.
is_present = 0x1000 in ih
This has a trivial fix by implementing __contains__
. I have implemented a fix and a test for it in my fork: https://github.com/sveinse/intelhex. Please give notice if a PR is wanted for this. https://github.com/sveinse/intelhex
When trying to run hexinfo.py on an intelhex image, I get the following:
File "c:\projects\test\venv\Scripts\hexinfo.py", line 63, in summarize_yaml
keys.sort()
AttributeError: 'dict_keys' object has no attribute 'sort'
> pip show intelhex -v
Name: intelhex
Version: 2.1
Summary: Python Intel Hex library
Home-page: https://pypi.python.org/pypi/IntelHex
Author: Alexander Belchenko
Author-email: [email protected]
License: BSD
Location: c:\projects\test\venv\lib\site-packages
Requires:
Metadata-Version: 2.0
Installer: pip
Classifiers:
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 3
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Telecommunications Industry
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Embedded Systems
Classifier: Topic :: Utilities
Entry-points:
> python -V
Python 3.5.2 :: Anaconda 4.1.1 (64-bit)
The readthedocs site doesn't appear to have any documentation for the hex2bin function (only the script which can be used from outside of Python is documented). Search didn't help (See Related issues).
Related issues:
Hey. I tried use hex2bin function im my code then tried your script and all the time I get this error:
Traceback (most recent call last): File "hex2.py", line 139, in <module> sys.exit(hex2bin(fin, fout, start, end, size, pad)) File "/usr/local/lib/python3.4/dist-packages/intelhex/__init__.py", line 1001, in hex2bin h.tobinfile(fout, start, end) File "/usr/local/lib/python3.4/dist-packages/intelhex/__init__.py", line 412, in tobinfile fobj.write(self._tobinstr_really(start, end, pad, size)) TypeError: must be str, not bytes
Script I use as python3 hex2bin Blinky.hex
installing w/ pip (7.1.2, setuptools 18.4) on python 2.7.10, the scripts were installed mode 0644 (i.e., not set executable).
Hello,
In the current version (2.3.0), the padding byte resets to 0xFF when a slice is taken:
In [14]: ih.padding
Out[14]: 0
In [15]: ih[:4].padding
Out[15]: 255
It would make sense to copy the padding byte to the new instance.
I am reading a PIC16 flash image from the target (oh instance IntelHex16bit), and merging it with a new image (nh instance IntelHex16bit) and get the following error:
nh.merge(oh[base:base+size], overlap='replace')
File "C:\Users\md84189\AppData\Local\Programs\Python\Python39\lib\site-packages\intelhex_init_.py", line 964, in getitem init.py", line 964, in getitem
addr1 = addr16 * 2
TypeError: unsupported operand type(s) for *: 'slice' and 'int'
If both instances are IntelHex or if nh instance is IntelHex and oh remains IntelHex16bit, the operation completes.
Note: Using Python 3.9.0, intelhex-2.3.0
I tried to load an HEX-Image from a TemporaryFile() with python 3.
Unfortunately the behaviour of TemporaryFile() differ from a normal file object.
Whereas file object read 'str' type, a TemporaryFile() read and writes 'bytes' type.
You can see the conversion in the code snippet below.
fobj = open('testimage.hex','r')
data = fobj.read()
ftmp = TemporaryFile()
ftmp.write(bytes(data,encoding='UTF-8'))
ftmp.seek(0)
ih.loadfile(ftmp, 'hex')
fobj.close()
ftmp.close()
Executing the code throws following exception:
decode(s, line)
File "D:\Programme\WinPython-64bit-3.4.3.3\python-3.4.3.amd64\lib\site-packages\intelhex_init_.py", line 111, in _decode_record
s = s.rstrip('\r\n')
TypeError: 'str' does not support the buffer interface
In ..._init_.py", line 111, in _decode_record, I inserted
if isinstance(s, bytes):
s = s.decode()
s = s.rstrip('\r\n')
before the s = s.rstrip('\r\n') to convert it to str.
Well, it solves my problem in python 3. I don't know if it is compatible with python 2.
What do you think? Is it applicable in future releases.
I use script hexmerge.py to merge two hex file, but after merging, I find the merged file size is larger than other tool.
For example, I have 3 files, 924k(0x8000000~0x8003AF0), 460k(0x8009000~0x80039210), 512k(0x8080000~0x80FFB30), first I use intelhex to merge, the merged file is 2.0M, than I use another tool named srecord, the merged file is 1.7M, but both of them work well.
I notice, the merged file used by intelhex like this:
:10000000288000205102000859020008ED1800085D
and srecord like this:
:20000000288000205102000859020008ED1800085D0200085F020008610200080000000012
I guess maybe there some format that intelhex used make it difference?
The ram and rom width of TI C28x DSP such as TMS320F28335 is 16 bits, and data word is big endian. I use command below to generate hex file from ELF file.
hex2000.exe --memwidth=16 --romwidth=16 --intel -o dat.hex firmware.out
Here is a segment of hex file of C28x. Word on address 0x3F2132 is 0x835E and word on address 0x3F2142 is 0x0000 and word on address 0x3F2152 is 0xAAAB.
:02000004003FBB
:20213200835E3F6C09083F71FA0B3F7453F83F7814BE3F7B3AAC3F7DC46D3F7EB10F3F7FEC
:2021420000003F8000004300F98341A20FDB3D49007F00000000BF00AAABBE2AAAAB3E2A74
:0C215200AAAB3D2A88893C088889BC089B
:00000001FF
When I use IntexHex16bit to parse the hex file, it raised AddressOverlapError. Can you add support for C28x? Thank you.
hexdiff.py version 2.2.1
adams@ubuntu:/mnt/f$ pip install intelhex
Requirement already satisfied: intelhex in /home/adams/.local/lib/python2.7/site-packages (2.2.1)
adams@ubuntu:/mnt/f$ python hexdiff.py device1.raw device2.raw
Traceback (most recent call last):
File "hexdiff.py", line 90, in <module>
sys.exit(main())
File "hexdiff.py", line 84, in main
ih1 = IntelHex(fname1)
File "/home/adams/.local/lib/python2.7/site-packages/intelhex/__init__.py", line 90, in __init__
self.loadhex(source)
File "/home/adams/.local/lib/python2.7/site-packages/intelhex/__init__.py", line 212, in loadhex
decode(s, line)
File "/home/adams/.local/lib/python2.7/site-packages/intelhex/__init__.py", line 123, in _decode_record
raise HexRecordError(line=line)
intelhex.HexRecordError: Hex file contains invalid record at line 1
adams@ubuntu:/mnt/f$ python
python python2.7 python2-config python3.7 python3-futurize python3-pasteurize python-config
python2 python2.7-config python3 python3.7m python3m python3-wsdump
adams@ubuntu:/mnt/f$ python3.7 hexdiff.py device1.raw device2.raw
Traceback (most recent call last):
File "hexdiff.py", line 90, in <module>
sys.exit(main())
File "hexdiff.py", line 83, in main
from intelhex import IntelHex, diff_dumps
ModuleNotFoundError: No module named 'intelhex'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.