nolze / msoffcrypto-tool Goto Github PK
View Code? Open in Web Editor NEWPython tool and library for decrypting and encrypting MS Office files using passwords or other keys
Home Page: https://msoffcrypto-tool.readthedocs.io/
License: MIT License
Python tool and library for decrypting and encrypting MS Office files using passwords or other keys
Home Page: https://msoffcrypto-tool.readthedocs.io/
License: MIT License
Edit: whups, I had a brainwave after posting this, and realized I was trying to overwrite the file while I had it open. Sorry to take up time.
I have a file protected with a company Sensitivity Label encrypted and am trying to open it, but get an error Unrecognized file format.
import msoffcrypto
protected = './protected.xlsx'
unprotected = './unprotected.xlsx'
with open(protected, 'rb') as f:
ms_file = OfficeFile(f)
print(ms_file)
And I get
FileFormatError: Unrecognized file format
Also, when checking .is_encrypted(), the file gives the same issue.
with open(protected, 'rb') as f:
ms_file = OfficeFile(f).is_encrypted()
print(ms_file)
The file is working and if I use Unencrypted reads fine. The file is not password protected, but just uses the sensitivity label as encryption.
I expected that is_encrypted should at least say True, and not just fail completely.
Not sure if the current implementation can even open these files with a sensitivity flag, even if you have the key
Thanks for adding support for ppt! Decrypting a simple password-protected ppt file (created by PowerPoint 2013) results in a successful decryption, however when opened by PowerPoint 2013 it launches in Protected View since "Office has detected a problem with this file".
According to Office, one of the reasons to open up in Protected View is:
File validation failure - When you see a message in Protected View that says "Office has detected a problem with this file. Editing it may harm your computer. Click for more details.", the file didnโt pass file validation. File validation scans file for security problems that can result from changes in the file structure.
Has anyone seen this? Would it help to add any more info?
Hi,
This software is great and has proven very useful to us to be able to convert encrypted office docs to pdf files.
I was just wondering if there was any plans to support MS-PPT (PowerPoint 2002, 2003, 2004) files? I know these are not currently supported so this isn't a bug as such. Just wondering about future plans?
We appreciate your help and effort on this project.
Thanks very much,
Stephen
Hello,
What would it take for the library to also support encryption of docx files? :)
import msoffcrypto
protected = './protected.xlsx'
unprotected = './unprotected.xlsx'
password = '12345'
file = open(protected, 'rb')
ms_file = msoffcrypto.OfficeFile(file).is_encrypted()
print(ms_file)
This return
True
, but when I try
file = open(unprotected, 'rb')
ms_file = msoffcrypto.OfficeFile(file).is_encrypted()
print(ms_file)
This give me
FileFormatError: Unencrypted document or unsupported file format
instead of False
I have an encrypted xlsx file, don't know which format and trying to decrypt it with:
file.load_key(password=args.password)
gives me:
..../msoffcrypto/format/ooxml.py", line 116, in load_key
raise AssertionError()
Key is not verified
Hi there,
We're looking at supporting encrypted files when accepting files for importing data into our application (currently we have to ask users to remove the password protection first).
This tool works great (thanks!) for decrypting on environments where python is available & the correct dependencies are installed - however I'd like to try and make our deployment as lean as possible by including a pre-compiled binary with no dependency on having Python already installed (we lack any strong Python experience in the team, so would like to avoid any problems that arise as a result).
I've had a play around with pyinstaller with limited success. It gave me an executable that I could use, however it was struggling with handling relative imports. The resulting executable was around 6MB which seems like a worthwhile sacrifice for the transportability.
I'm too much of a Python noob to figure out the pyinstaller issues, but I'm pretty confident it can be accomplished with some help!
Would you consider providing standalone binaries/executables, or perhaps even an 'official' Docker image with your releases?
Many thanks,
Dec
Hi,
Any idea how Office 95 word documents should be decrypted?
Thanks,
Bart
Hello,
Installed using pip on Ubuntu:
root@Laptop:~# python /usr/local/bin/msoffcrypto-tool -p passwd xxx.xls out.xls
Traceback (most recent call last):
File "/usr/local/bin/msoffcrypto-tool", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/msoffcrypto/__main__.py", line 67, in main
file = OfficeFile(args.infile)
File "/usr/local/lib/python2.7/dist-packages/msoffcrypto/__init__.py", line 30, in OfficeFile
raise AssertionError("Unrecognized file format")
AssertionError: Unrecognized file format
Thanks for your help!
It would be better to raise a more specific exception than Exception
- in fact I believe Exception
should not be raised and it's real use is as a catch all for try/catch.
By using something more appropriate, eg ValueError(?), or implementing our own error types we can catch errors better.
excuse the naming but an example is below:
MsOffCryptoBaseException(Exception):
pass
IncorrectPasswordException(MsOffCryptoBaseException):
pass
Currently I need to inspect the exception message to determine the cause of failure. I believe raising a specific error is a much nicer pattern and will not break compatibility with previous code as catch Exception will still catch this.
I found this repo after I made my own ad-hoc implementation. The code here is well-structured. However, it seems you do not check if the key supplied by the user is correct before decrypting the file. Also, the hmac of the encrypted payload is not verified.
I have implemented them according to the spec. When I have some time, I will submit a pull request with these two features added.
As for Agile, one can verify if the key is correct using "encryptedVerifierHashInput" and "encryptedVerifierHashValue". And verify the hmac of the payload using "encryptedHmacKey" and "encryptedHmacValue". I did not check other cryptos supported by this repo.
Hi,
I would like to check if an office file is encrypted by using the Msoffcrypto-tool without using the CLI. Meaning, using it as a package, is this possible?
I can't seem to find any documentation on functionality / methods other than with the CLI?
Something like this:
inputPath = "tests/inputs/plain.doc"
file = msoffcrypto.OfficeFile(open(inputPath, "rb"))
file_encr = msoffcrypto.file.is_encrypted()
print(file_encr)
Thanks!
Traceback (most recent call last):
File "c:/Project/Human_Machine/old_task/word_activity.py", line 13, in
file.decrypt(f)
File "C:\Users\hari\AppData\Local\Programs\Python\Python36\lib\site-packages\msoffcrypto\format\ooxml.py", line 198, in decrypt
if not zipfile.is_zipfile(io.BytesIO(obuf)):
UnboundLocalError: local variable 'obuf' referenced before assignment
`
import msoffcrypto
encrypted = open(
'C:\Project\Human_Machine\old_task\word_text.docx',
"rb")
file = msoffcrypto.OfficeFile(encrypted)
file.load_key(password="Passw0rd") # Use password
with open("decrypted.docx", "wb") as f:
file.decrypt(f)
print("encypt completed")
encrypted.close()
`
After I run the above code one files get created of 0kb.
if I input an .xls file, decrypted file is .xlsx format? or I can select .xls / .xlsx?
when I test according to README.md, the error was raised.
the code in ipython is :
In [1]: import msoffcrypto
In [2]: file = msoffcrypto.OfficeFile(open("blnr_cyjl.odt", "rb"))
In [3]: file.load_key(password="#rx@q&ft47*1p3zkeg")
In [4]: file.decrypt(open("blnr_cyjl.odt", "wb"))
---------------------------------------------------------------------------
UnboundLocalError Traceback (most recent call last)
<ipython-input-4-f463435a37e0> in <module>()
----> 1 file.decrypt(open("blnr_cyjl.odt", "wb"))
/Library/Python/2.7/site-packages/msoffcrypto/format/ooxml.pyc in decrypt(self, ofile)
150
151 # If the file is successfully decrypted, there must be a valid OOXML file, i.e. a valid zip file
--> 152 if not zipfile.is_zipfile(io.BytesIO(obuf)):
153 raise Exception("The file could not be decrypted with this password")
154
UnboundLocalError: local variable 'obuf' referenced before assignment
In [5]:
Since v0.45, olefile can overwrite streams of any size. This enables simplifying some decryption routines.
When evaluating a password protected PPT there is no error and it looks as though everything happens as expected
This (run against a known password protected PPT):
filePPT = r'F:\Decepticons-office-pw.ppt'
print(msoffcrypto.OfficeFile(open(filePPT, "rb")).is_encrypted())
if (msoffcrypto.OfficeFile(open(filePPT, "rb")).is_encrypted()) == 0:
print("File is not password protected.")
else:
print("File is password protected.")
Produces:
C:\Python\Python38-32\python.exe F:/TestOfficePassword.py
True
File is password protected.
Process finished with exit code 0
This (run againsta a known unprotected PPT):
filePPT = r'F:\Decepticons-office.ppt'
print(msoffcrypto.OfficeFile(open(filePPT, "rb")).is_encrypted())
if (msoffcrypto.OfficeFile(open(filePPT, "rb")).is_encrypted()) == 0:
print("File is not password protected.")
else:
print("File is password protected.")
Produces:
C:\Python\Python38-32\python.exe F:/TestOfficePassword.py
Traceback (most recent call last):
File "F:/TestOfficePassword.py", line 12, in <module>
print(msoffcrypto.OfficeFile(open(filePPT, "rb")).is_encrypted())
File "C:\Python\Python38-32\lib\site-packages\msoffcrypto\format\ppt97.py", line 756, in is_encrypted
usereditatom = _parseUserEditAtom(self.data.powerpointdocument)
File "C:\Python\Python38-32\lib\site-packages\msoffcrypto\format\ppt97.py", line 234, in _parseUserEditAtom
encryptSessionPersistIdRef, = unpack("<I", blob.read(4))
struct.error: unpack requires a buffer of 4 bytes
Process finished with exit code 1
PPTX appear to function correctly for both protected and unprotected files.
Thank you in advance for any help.
Unable to decrypt an office '97 document. The decryption fails here: https://github.com/nolze/msoffcrypto-tool/blob/master/msoffcrypto/format/doc97.py#L392
Document file is attached to the email found here: https://www.malware-traffic-analysis.net/2018/08/21/index2.html
Please contact me with questions, and I can provide the raw file for your convenience.
Any idea what this error means and how to work fix it?
Generated from the following code:
import msoffcrypto
file = msoffcrypto.OfficeFile(open("/Users/path/to/my/file.docx", "rb"))
file.load_key(password="password")
file.decrypt(open("/Users/path/to/my/file.docx", "wb"))
I can't seem to get this library to work with my XLS File.
import io
import pandas as pd
PATH = src_dir+"/"+files[0]
print(PATH)
decrypted = io.BytesIO()
with open(PATH, "rb") as f:
file = msoffcrypto.OfficeFile(f)
print(file.is_encrypted())
file.load_key(password="Password") # Use password
file.decrypt(decrypted)
df = pd.read_excel(decrypted)
print(df)
I'm getting InvalidKeyError: Failed to verify password. Is this due to XLS being in experimental stage?
with open("encrypted.xlsx", "rb") as f:
file = msoffcrypto.OfficeFile(f)
file.load_key(password="Passw0rd") # Use password
print( file.load_key(password="Passw0rd"))
file.decrypt(decrypted)
df = pd.read_excel(decrypted)
This print always shows None even if the password is correct.
I would like to display a message if the password is incorrect:
with open("encrypted.xlsx", "rb") as f:
file = msoffcrypto.OfficeFile(f)
if != None:
file.load_key(password="Passw0rd") # Use password
else:
console.alert('The password is not correct, try again')
file.decrypt(decrypted)
df = pd.read_excel(decrypted)
Hello,
The file .../site-packages/msoffcrypto/format/xls97.py:
class Xls97File has a problem in "decrypt" function.
Sometimes it fails on "if self.type ==" when self has NO "type" attribute...
A nasty workaround is to try it within Try catch, before real evaluation and return if it fails, but it would be nice to prevent it in code which populates the type...
Thx
file.load_key(password=str(pwd), verify_password=True)
I success decrypt one xls file , but when try another xls file , raise this error .
because of same ilename extension different format ?
how to fix it thanks
Hello,
I understand that this is potentially out of scope for the project, but considering the existence of OfficeFile.is_encrypted()
I feel this would tie its usage up nicely.
I'll explain a use case via example:
I am using this to load up a set of usually-encrypted Excel files into pandas
, this is great, except a handful of these Excel files have randomly have not been password protected. I don't actually care whether or not they have a password, I just want to put them all into dataframes.
Right now, the argument I pass to pandas.read_excel()
is either a non-protected Excel file's Path
, or a BytesIO
objected retrieved using this library.
This is fine but it has resulted in this messy function:
def decrypt_office_file(file: Path, password: str = None) -> Union[io.BytesIO, Path]:
decrypted_file = io.BytesIO()
with open(file, 'rb') as f:
office_file = msoffcrypto.OfficeFile(f)
if office_file.is_encrypted():
office_file.load_key(password=password)
office_file.decrypt(decrypted_file)
else:
decrypted_file = file
return decrypted_file
excel_file = decrypt_office_file("my_file.xlsx")
df = pd.read_excel(excel_file, ...)
And then I just have to hope everything downstream is cool with taking either a BytesIO
or a str
/Path
, which is okay for pandas but I imagine is less okay for other libraries/use cases.
I'm not sure how it would be best to insert the functionality, but something like OfficeFile.to_bytes()
(I'm sure there are better ideas for function names available) would be great, then we can have consistent return types.
I also find it really odd that .decrypt()
takes the object you want to inject the file into as an argument, rather than returning a BytesIO object? It makes following the code flow feel awkward to me, but that's an issue for another day!
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/msoffcrypto/format/xls97.py", line 644, in is_encrypted
if not workbook.has_record(recordNameNum["FilePass"]):
File "/usr/local/lib/python3.10/dist-packages/msoffcrypto/format/xls97.py", line 418, in has_record
num, size = unpack("<HH", h)
struct.error: unpack requires a buffer of 4 bytes
Unfortunately I cannot share the file that triggers this exception.
But looking at xls97.py
line 418, the code assumes that self.data.read(4)
returns either exactly 4 bytes or nothing:
msoffcrypto-tool/msoffcrypto/format/xls97.py
Lines 411 to 423 in f727b42
I'm not sure what self.data
there is but normally read(n)
is guaranteed to return at most 4 bytes, not exactly 4 bytes.
Excel file name with (), for example, test(1).xlsx will always return no password protection (even password protected).
Just wanted to say thank you. I thought I was going to have to go down a rabbit hole trying to decrypt a .xls file on a mac, and this took 30 seconds. Appreciate your work here!
Office 16 test case
Tried it via a script and it throws the following exception:
totalSize = unpack('<I', ibuf.read(4))[0]
struct.error: unpack requires a buffer of 4 bytes
On line 7 of init.py, the module version is retrieved.
__version__ = pkg_resources.get_distribution("msoffcrypto-tool").version
This does not work when the module is used as a part of a compiled program as msoffcrypto is the name of the actual import and is thus what is compiled.
I have worked around this temporarily by modifying this line to just set a value.
__version__ = 1
Steps to recreate
Attempt to compile a program using msoffcrypto (I used auto-py-to-exe)
Run the created executable from a terminal.
Is there any plan to support decryption of XORObfuscation algorithm?
Hello,
I have noticed that the file msoffcrypto/method/xor_obfuscation.py is empty - is that OK?
Best regards
Michal Ambroz
The BaseOfficeFile defines three functions that can be implemented. If we look at load_key(), doc97, ppt97 and xls97 take the same arguments and behave similarly. Looking at ooxml, it takes more arguments and does not validate the password (if provided) by default. As I want to validate the password, I cannot call the function load_key() in an agnostic way.
Do you think it would be valuable to catch extra parameters in all implementations of load_key() and decrypt() with *args + **kwargs to make it more uniform?
I can see the downside of a user passing parameters that are ignored, and not understanding why it does nothing, so I do not have strong feelings on this suggestion.
Thank you for your time! ๐
Hello,
We maintain a small tool for extracting embedded objects in OneNote documents here:
https://github.com/volexity/threat-intel/tree/main/tools/one-extract
One of the things that is supported is the extracted of password-protected objects, for this we were using the following method from your library:
https://github.com/nolze/msoffcrypto-tool/blob/master/msoffcrypto/method/ecma376_agile.py
One user noticed that at each 4096 byte boundary there were 16 bytes of invalid data that were being added and provided test case files illustrating the issue:
We have added a temporary fix here:
It's not 100% clear why our temporary fix (setting the SEGMENT_LENGTH
value to a size greater than the size of the file) works, some possible hypotheses are:
emca376_agile.py
code.emca376_agile.py
, is different (hard to figure out if this is the case, since there isn't a lot of material online describing it in detail.Cheers,
Tom
Pulling the code from PyPI gives different code to that in the repository.
As an example, the file msoffcrypto/format/ooxml.py
has a class (OOXMLFile
) with a method load_key
. From the code in this repository, I would expect that method to accept a keyword argument verify_password
. The version of the file available on PyPI as version 4.10.2 (and 4.10.1, but that's all I've checked) does not.
Similarly, the decrypt
method of the same class is missing its verify_integrity
argument.
(venv) $ pip install msoffcrypto-tool
Collecting msoffcrypto-tool
Using cached msoffcrypto-tool-4.11.0.tar.gz (211 kB)
Collecting olefile>=0.45
Using cached olefile-0.46.zip (112 kB)
Collecting cryptography>=2.3
Using cached cryptography-3.2.1.tar.gz (540 kB)
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: /Users/gordio/MyProject/venv/bin/python /Users/gordio/MyProject/venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-build-env-xdyteey0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"''
cwd: None
Complete output (118 lines):
Collecting setuptools>=40.6.0
Using cached setuptools-50.3.2-py3-none-any.whl (785 kB)
Collecting wheel
Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Collecting cffi!=1.11.3,>=1.8
Using cached cffi-1.14.3.tar.gz (470 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Building wheels for collected packages: cffi
Building wheel for cffi (setup.py): started
Building wheel for cffi (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /Users/gordio/MyProject/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-wheel-_q0mb1sa
cwd: /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/
Complete output (45 lines):
/bin/sh: brew: command not found
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.14.6-x86_64-3.8
creating build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/backend_ctypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/error.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/setuptools_ext.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/cffi_opcode.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/vengine_gen.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/pkgconfig.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/model.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/ffiplatform.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/api.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/vengine_cpy.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/commontypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/lock.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/recompiler.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/cparser.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/verifier.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/_cffi_include.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/parse_c_type.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/_embedding.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/_cffi_errors.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
building '_cffi_backend' extension
creating build/temp.macosx-10.14.6-x86_64-3.8
creating build/temp.macosx-10.14.6-x86_64-3.8/c
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -I/usr/local/opt/libffi/include -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/ffi -I/Users/gordio/MyProject/venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c c/_cffi_backend.c -o build/temp.macosx-10.14.6-x86_64-3.8/c/_cffi_backend.o
c/_cffi_backend.c:5854:2: error: Apple Arm64 ABI requires ffi_prep_cif_var
#error Apple Arm64 ABI requires ffi_prep_cif_var
^
c/_cffi_backend.c:6304:9: warning: 'ffi_prep_closure' is deprecated [-Wdeprecated-declarations]
if (ffi_prep_closure(closure, &cif_descr->cif,
^
/usr/local/opt/libffi/include/ffi.h:341:18: note: 'ffi_prep_closure' has been explicitly marked deprecated here
__attribute__((deprecated))
^
1 warning and 1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: setuptools, wheel, pycparser, cffi
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /Users/gordio/MyProject/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-record-x2d15bda/install-record.txt --single-version-externally-managed --prefix /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-build-env-xdyteey0/overlay --compile --install-headers /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-build-env-xdyteey0/overlay/include/site/python3.8/cffi
cwd: /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/
Complete output (45 lines):
/bin/sh: brew: command not found
running install
running build
running build_py
creating build
creating build/lib.macosx-10.14.6-x86_64-3.8
creating build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/backend_ctypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/error.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/setuptools_ext.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/cffi_opcode.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/vengine_gen.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/pkgconfig.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/model.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/ffiplatform.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/api.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/vengine_cpy.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/commontypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/lock.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/recompiler.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/cparser.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/verifier.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/_cffi_include.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/parse_c_type.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/_embedding.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
copying cffi/_cffi_errors.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
building '_cffi_backend' extension
creating build/temp.macosx-10.14.6-x86_64-3.8
creating build/temp.macosx-10.14.6-x86_64-3.8/c
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -I/usr/local/opt/libffi/include -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/ffi -I/Users/gordio/MyProject/venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c c/_cffi_backend.c -o build/temp.macosx-10.14.6-x86_64-3.8/c/_cffi_backend.o
c/_cffi_backend.c:5854:2: error: Apple Arm64 ABI requires ffi_prep_cif_var
#error Apple Arm64 ABI requires ffi_prep_cif_var
^
c/_cffi_backend.c:6304:9: warning: 'ffi_prep_closure' is deprecated [-Wdeprecated-declarations]
if (ffi_prep_closure(closure, &cif_descr->cif,
^
/usr/local/opt/libffi/include/ffi.h:341:18: note: 'ffi_prep_closure' has been explicitly marked deprecated here
__attribute__((deprecated))
^
1 warning and 1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/gordio/MyProject/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-install-2_yt854r/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-record-x2d15bda/install-record.txt --single-version-externally-managed --prefix /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-build-env-xdyteey0/overlay --compile --install-headers /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-build-env-xdyteey0/overlay/include/site/python3.8/cffi Check the logs for full command output.
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/gordio/MyProject/venv/bin/python /Users/gordio/MyProject/venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/f0/js5ybgr96yx8th3mk0l8148m0000gn/T/pip-build-env-xdyteey0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"'' Check the logs for full command output.
Hello,
I am unzipping excel files and then trying to open them to store them in dataframes but I get a permission error that makes me think I am not closing something properly any ideas? Here is a sample of the code:
def mistubishi_password(self, password, x, attachments):
for att in attachments:
with ZipFile(self.DIR_SBD + att.FileName) as zipObj:
listOfFileNames = zipObj.namelist()
print(listOfFileNames)
for file in listOfFileNames:
file_extension = file.split(".")[1]
if file_extension.lower() == "xls":
if(("[Trial Balance_JPY]" in file) | ("[Traditional Fund Structure NAV]" in file)):
goodfile = file
zipObj.extract(goodfile, path = self.DIR_SBD + goodfile, pwd=password.encode())
if("[Trial Balance_JPY]" in goodfile):
zipObj.close()
protfile = msoffcrypto.OfficeFile(open(self.DIR_SBD + goodfile, "rb"))
protfile.load_key(password=password)
decrypted = io.BytesIO()
protfile.decrypt(decrypted)
df = pd.read_excel(decrypted)
print(df)
return None
Here is the error message I am getting:
July 29, 2021 NAV - MY-AMD Global Multi Asset Fund 1327014 [PW]
15%|โโ | 88/584 [00:43<02:21, 3.50it/s]['[Traditional Fund Structure NAV] [1327014].pdf', '[Traditional Fund Structure NAV] [1327014].xls']
15%|โโ | 88/584 [00:43<04:07, 2.01it/s]
Traceback (most recent call last):
File "C:/Users/giraudl/PycharmProjects/Automatization_PDP/main.py", line 501, in <module>
GestionBoitePDP.get_mail_print_to_csv()
File "C:/Users/giraudl/PycharmProjects/Automatization_PDP/main.py", line 483, in get_mail_print_to_csv
self.mistubishi_password(self.password_1327014, x, attachments)
File "C:/Users/giraudl/PycharmProjects/Automatization_PDP/main.py", line 110, in mistubishi_password
file = msoffcrypto.OfficeFile(open(self.DIR_SBD + goodfile, "rb"))
PermissionError: [Errno 13] Permission denied: 'C:/Users/giraudl/PycharmProjects/Automatization_PDP/attachments/[Traditional Fund Structure NAV] [1327014].xls
I'm having an issue with using this package in a virtual environment in Python 3.11. I noticed the pkg_resources call to pull the versioning when msoffcrypto-tool runs the init.py is wrong.
It seems this should now be done using the importlib_metadata, which is included since Python 3.8. I've updated the module locally to import with this method and now do not get any distribution errors:
try:
import importlib_metadata
except ImportError:
import importlib.metadata as importlib_metadata
__version__ = importlib_metadata.version("msoffcrypto-tool")
Confirmed here https://setuptools.pypa.io/en/latest/pkg_resources.html, pkg_resources is deprecated.
When trying to test for encryption on an unencrypted Excel 97-2003 file:
stewartwebb@shrinkpad:~/Documents$ msoffcrypto-tool -t ./test.xls
Traceback (most recent call last):
File "/home/stewartwebb/.virtualenvs/venv/bin/msoffcrypto-tool", line 11, in <module>
sys.exit(main())
File "/home/stewartwebb/.virtualenvs/venv/local/lib/python2.7/site-packages/msoffcrypto/__main__.py", line 50, in main
if not is_encrypted(args.infile):
File "/home/stewartwebb/.virtualenvs/venv/local/lib/python2.7/site-packages/msoffcrypto/__main__.py", line 34, in is_encrypted
return file.is_encrypted()
File "/home/stewartwebb/.virtualenvs/venv/local/lib/python2.7/site-packages/msoffcrypto/format/xls97.py", line 583, in is_encrypted
num, size = workbook.skip_to(47)
TypeError: 'NoneType' object is not iterable
stewartwebb@shrinkpad:~/Documents$
I was able to reproduce this by making a new spreadsheet in OpenOffice Calc, putting some random text in a few cells in all three default sheets, the saving and exporting as an Excel 97/2003 file (.xls).
Can't seem to see what would be causing that error - would it be anything to do with me running Python 2?
Hi there
Thanks for your work on this package. I've had success using your cli tool and as a library, but the 'in-memory' example doesn't work for me.
>>> df = pd.read_excel(decrypted) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/user/anaconda/lib/python3.5/site-packages/pandas/io/excel.py", line 170, in read_excel io = ExcelFile(io, engine=engine) File "/Users/user/anaconda/lib/python3.5/site-packages/pandas/io/excel.py", line 225, in __init__ self.book = xlrd.open_workbook(file_contents=data) File "/Users/user/anaconda/lib/python3.5/site-packages/xlrd/__init__.py", line 395, in open_workbook with open(filename, "rb") as f: TypeError: invalid file: None
Unfortunately I can't share the file for testing, other than to say it is working fine for the other methods. I can't see where this aspect of the package is getting tested. Do you have any suggestions?
Thanks
PS C:> msoffcrypto-tool -p 123456 .\test.xlsx
Traceback (most recent call last):
File "D:\Program Files\Anaconda3\Scripts\msoffcrypto-tool-script.py", line 11, in
load_entry_point('msoffcrypto-tool==4.6.1', 'console_scripts', 'msoffcrypto-tool')()
File "d:\program files\anaconda3\lib\site-packages\msoffcrypto_tool-4.6.1-py3.6.egg\msoffcrypto_main_.py", line 58, in main
AssertionError: Not OLE file
Although it makes for a bit more inelegant code, splitting makekey_from_password
in ecma376_*.py
into 2 versions - one for sha1 and another for sha512 improves performance by around 12%. This is because of the tight loop "spinning" the hash, where calling _hashCalc with the same second argument is unnecessary.
I don't know if there is some trick to make CPython optimize it better, I'll look into making a single call before the loop. I'm just trowing it out there.
Here's my benchmark:
Results:
$ python2 orig.py
8291acf18cc84888a751f426a89103d1
5.23682308197
$ python2 opt.py
8291acf18cc84888a751f426a89103d1
4.69491386414
$ python2 orig.py
8291acf18cc84888a751f426a89103d1
5.24634695053
$ python2 opt.py
8291acf18cc84888a751f426a89103d1
4.62075614929
orig.py takes 5.2 seconds, opt.py - 4.6.
Essentially as it is right now (orig.py):
import hashlib, functools, io
from struct import pack, unpack
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
def _hashCalc(i, algorithm):
if algorithm == "SHA512":
return hashlib.sha512(i)
else:
return hashlib.sha1(i)
def makekey_from_password(password, saltValue, hashAlgorithm, encryptedKeyValue, spinValue, keyBits):
block3 = bytearray([0x14, 0x6e, 0x0b, 0xe7, 0xab, 0xac, 0xd0, 0xd6])
# Initial round sha512(salt + password)
#h = _hashCalc(saltValue + password.encode("UTF-16LE"), hashAlgorithm)
h = _hashCalc(saltValue + password.encode("UTF-16LE"), hashAlgorithm) #STJO
# Iteration of 0 -> spincount-1; hash = sha512(iterator + hash)
for i in range(0, spinValue, 1):
h = _hashCalc(pack("<I", i) + h.digest(), hashAlgorithm)
h2 = _hashCalc(h.digest() + block3, hashAlgorithm)
# Needed to truncate skey to bitsize
skey3 = h2.digest()[:keyBits // 8]
# AES encrypt the encryptedKeyValue with the skey and salt to get secret key
aes = Cipher(algorithms.AES(skey3), modes.CBC(saltValue), backend=default_backend())
decryptor = aes.decryptor()
skey = decryptor.update(encryptedKeyValue) + decryptor.finalize()
return skey
import time
password = "CSIS062027"
slt = "084cfc2bfae6ac80bdd5eaabdd0b977b"
enc = "9cc0d8a4a4b65322c46dfed4e4cd0b51"
saltValue = slt.decode("hex")
encValue = enc.decode("hex")
begin = time.time()
ret = ""
for i in range(50):
ret = makekey_from_password(password, saltValue, "SHA1", encValue, 100000, 128)
end = time.time()
print ret.encode("hex")
print end - begin
The "optimized" version (opt.py):
import hashlib, functools, io
from struct import pack, unpack
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
def makekey_from_password(password, saltValue, hashAlgorithm, encryptedKeyValue, spinValue, keyBits):
block3 = bytearray([0x14, 0x6e, 0x0b, 0xe7, 0xab, 0xac, 0xd0, 0xd6])
# Initial round sha512(salt + password)
#h = _hashCalc(saltValue + password.encode("UTF-16LE"), hashAlgorithm)
h = hashlib.sha1(saltValue + password.encode("UTF-16LE")) #STJO
# Iteration of 0 -> spincount-1; hash = sha512(iterator + hash)
for i in range(0, spinValue, 1):
h = hashlib.sha1(pack("<I", i) + h.digest())
h2 = hashlib.sha1(h.digest() + block3)
# Needed to truncate skey to bitsize
skey3 = h2.digest()[:keyBits // 8]
# AES encrypt the encryptedKeyValue with the skey and salt to get secret key
aes = Cipher(algorithms.AES(skey3), modes.CBC(saltValue), backend=default_backend())
decryptor = aes.decryptor()
skey = decryptor.update(encryptedKeyValue) + decryptor.finalize()
return skey
import time
password = "CSIS062027"
slt = "084cfc2bfae6ac80bdd5eaabdd0b977b"
enc = "9cc0d8a4a4b65322c46dfed4e4cd0b51"
saltValue = slt.decode("hex")
encValue = enc.decode("hex")
begin = time.time()
ret = ""
for i in range(50):
ret = makekey_from_password(password, saltValue, "SHA1", encValue, 100000, 128)
end = time.time()
print ret.encode("hex")
print end - begin
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.