pycontribs / tendo Goto Github PK
View Code? Open in Web Editor NEWOfficial repository of python tendo library, always welcoming new contributions.
License: Other
Official repository of python tendo library, always welcoming new contributions.
License: Other
Hello
I've just made the same issue to one project (PyBitMessage) that using your idea. The thing I want to talk aboue is file singleton.py with singleinstance class especially.
The design of this class is incorrect because of two things as below:
__del__
method will ever called. So it can easily lead to situation when lock file remains in system after app shutdown.try..except
block in __del__
method which would not work ever because, as stated in python doc, any exception that occurs while running __del__
method will be ignored and just printed a warning in stderr instead.Thus, I'd suggest to either make some other method like release()
that should be called directly or redesign this class\idea.
Thanks,
Constantine
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Hello,
It would be very cool to have a new release for the SingleInstance(lockfile)
new feature :)
after the program has been closed the single instance is staying alive preventing a new instance being opened.
The task has had be ending within task manager to run it again
While testing the script that uses SingleInstance, I noticed that log messagaes from SingleInstance are printed twice. I think this behavior relates to the StreamHandler added at line
Line 144 in b1e08eb
I would suggest moving this line into the 'if name == "main":', so that the StreamHandler will be created and used only within the unit tests as I think it was expected.
Installed tendo using pip install tendo. Package installed successfully.
>>> import tendo
>>> me = SingleInstance()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'SingleInstance' is not defined
>>> dir(tendo)
['VersionInfo', '__all__', '__author__', '__builtins__', '__cached__', '__copyright__', '__doc__', '__email__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__status__', '__version__', '_v', 'absolute_import', 'sys', 'version_info']
>>> tendo.__version__
'0.2.15'
>>>
The official documentation states that Tendo is compatible with Python 2.7 - 3.6. Are there any known issues using it with Python 3.7?
Line 53 in d3b0e9e
Is this non blocking I think we need to add os.O_NONBLOCK here as well similar to what we are doing for non windows versions.
I ran across this library while searching for a library to provide me with fs locking mechanism to run program once.
However, the script uses __del__
method, which is incorrect: https://github.com/pycontribs/tendo/blob/master/tendo/singleton.py#L71
From python docs:
It is not guaranteed that del() methods are called for objects that still exist when the interpreter exits.
Which would cause it to be locked forever.
This was issued by an output from the 'tree' command, possibly because it outputs colorized text. The colorer's error was:
File "/usr/lib/python2.7/site-packages/tendo/colorer.py", line 125, in new
except e:
NameError: global name 'e' is not defined"
which should be instead:
File "/usr/lib/python2.7/site-packages/tendo/colorer.py", line 127, in new
raise e
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 50: ordinal not in range(128)"
Fixed by this patch:
diff --git a/usr/lib/python2.7/site-packages/tendo/colorer.py.orig b/usr/lib/python2.7/site-packages/tendo/colorer.py
index 91f53c4..13a2dcb 100644
--- a/usr/lib/python2.7/site-packages/tendo/colorer.py.orig
+++ b/usr/lib/python2.7/site-packages/tendo/colorer.py
@@ -122,7 +122,7 @@ if (hasattr(sys.stderr, "isatty") and sys.stderr.isatty()) or \
color = '\x1b[0m' # normal
try:
new_args[1].msg = color + unicode(new_args[1].msg) + '\x1b[0m' # normal
- except e:
+ except Exception as e:
print type(new_args[1].msg)
raise e
# print "after"
is it possible to quiet the error message for singleton.py in a python script without editing the source? just so I can still get error messages from a cronjob if required
I need to run the system2 function together with certbot on windows and I have command like this on GNU/Linux:
sudo certbot certonly --manual --expand --manual-public-ip-logging-ok \
--preferred-challenges http -n \
-d <LIST OF COMMA SEPARATED DOMAINS AND SUBDOMAINS>\
--manual-auth-hook ./cert.py --agree-tos --email <EMAIL ADRESS> 2>&1 | tee $output
I'm using 2>&1 because certbot print some output to stderr but on windows I can use command with 2>&1.
Some may want to catch the exception instead and not quit. Perhaps the logger message should not include 'quitting' in the message.
Line 54 in 56c9a6f
On win32 logger.error is used else logger.warning is used. Perhaps select the same for both platforms?
Line 54 in 56c9a6f
Hi,
It seems the singleton is not blocking other instances if the script is called with different paths.
e.g.
one instance is run by
./script.py
and another instance is run by
/path/to/script/script.py
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Hi,
When attempting to implement tendo to make a simple Python script a singleton, the following blocking error occurs. Not sure of the cause, any help very welcomed.
File "/usr/local/lib/python3.6/site-packages/tendo/singleton.py", line 64, in __init__
fcntl.lockf(self.fp, fcntl.LOCK_EX | fcntl.LOCK_NB)
BlockingIOError: [Errno 11] Resource temporarily unavailable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/fs_api_cron/cron.py", line 10, in <module>
me = singleton.SingleInstance() # will sys.exit(-1) if other instance is running
File "/usr/local/lib/python3.6/site-packages/tendo/singleton.py", line 68, in __init__
raise SingleInstanceException()
tendo.singleton.SingleInstanceException```
Hello,
I am trying to run a Flask App in which tendo was used, I have installed tendo successfully but when I try to run the app I get the error below:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/macbook/Desktop/securityPI/prgms/webPrgm.py", line 155, in
me = singleton.SingleInstance()
File "/Users/macbook/Desktop/securityPI/pi/lib/python3.7/site-packages/tendo/singleton.py", line 70, in init
raise SingleInstanceException()
tendo.singleton.SingleInstanceException
I am using a Mac running macOS Catalina 10.15 and python3.7
I've added tendo to buildroot, but I get an installation error, and I'm not sure whether or not the problem lies with the tendo installer or elsewhere.
Buildroot gives me back (I think from tendo's setup.py) "distutils.errors.DistutilsError: Could not find suitable distribution fro Requirement.parse('six').
Initially I thought that perhaps the "six" package is a dependency, but after adding this to buildroot, I still get the error message. Even tried installing it locally, but no luck.
The latest change to the build process introduced a dependency on wstools and it is not listed in the dependencies in the setup.py, so it caused our scripts to fail until we manually pip installed wstools.
I put the example code (from singleton module) at the top of my script and it worked as expected. But when I converted it to .exe with PyInstaller the .exe also works fine and prevents multiple instances of my program, except there are some windows flashing before the program starts.
I guess this is the multiprocessing issue as always it flashes new windows in .exe files. So I removed the multiprocessing imports and usages from singleton.py and the window flashing was gone.
I think someone should prevent unitTests from running multiprocessing in .exe mode.
I am looking for maintainers/core to join this project as I an personally not using it anymore.
I can still keep providing CI/CD support for them.
Latest version on repo is 0.4.0, but on PyPi it is 0.3.0.
How/when will it be released there?
Windows 7
Python 3.6
tendo 0.2.12
Pyinstaller 3.4.dev0+b31794cf1
Hey all,
after updating tendo I created an .exe with Pyinstaller.
Trying to run my .exe gives the following error-message:
[WARNING] stderr: Traceback (most recent call last):
[WARNING] stderr: File "site-packages\pbr\version.py", line 442, in get_version_from_pkg_resources
[WARNING] stderr: File "site-packages\pkg_resources_init.py", line 344, in get_provider
[WARNING] stderr: File "site-packages\pkg_resources_init_.py", line 892, in require
[WARNING] stderr: File "site-packages\pkg_resources_init_.py", line 778, in resolve
[WARNING] stderr: pkg_resources.DistributionNotFound: The 'tendo' distribution was not found and is required by the application
[WARNING] stderr:
[WARNING] stderr: During handling of the above exception, another exception occurred:
[WARNING] stderr:
[WARNING] stderr: Traceback (most recent call last):
[WARNING] stderr: File "to_exe\main.py", line 79, in
[WARNING] stderr: from tendo import singleton
[WARNING] stderr: File "", line 971, in _find_and_load
[WARNING] stderr: File "", line 955, in _find_and_load_unlocked
[WARNING] stderr: File "", line 665, in load_unlocked
[WARNING] stderr: File "c:\users\mirk\appdata\local\programs\python\python36\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 628, in exec_module
[WARNING] stderr: exec(bytecode, module.dict)
[WARNING] stderr: File "site-packages\tendo_init.py", line 10, in
[WARNING] stderr: File "site-packages\pbr\version.py", line 462, in semantic_version
[WARNING] stderr: File "site-packages\pbr\version.py", line 449, in _get_version_from_pkg_resources
[WARNING] stderr: File "site-packages\pbr\packaging.py", line 816, in get_version
[WARNING] stderr: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name tendo was given, but was not able to be found.
Hi,
I found an example "single instance of program" on https://stackoverflow.com/a/1265445
I ran the script and got an error.
python3 test.py
#
import sys
from tendo import singleton
try:
me = singleton.SingleInstance()
except:
sys.exit()
Exception ignored in: <bound method SingleInstance.del of <tendo.singleton.SingleInstance object at 0x7f4cb68f3f28>>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tendo/singleton.py", line 70, in del
ModuleNotFoundError: import of os halted; None in sys.modules
Ubuntu 18.04.1 LTS, Python 3.6.6
I think
line 67: "except Exception, e:"
should be
"except Exception as e:"
96: f = open(os.path.join(self.dir, "tests/utf8.txt"), "rU")
107: f = open(os.path.join(self.dir, "tests/utf8-invalid.txt"), "rU")
When I make a Python package with setuptools
, then install it in a virtualenv and run it, I get this error:
Exception ImportError: 'No module named sys' in <bound method SingleInstance.__del__ of
<tendo.singleton.SingleInstance instance at 0x108618488>> ignored
Here are the project files:
#!/usr/bin/env python
# setup.py
from setuptools import setup
setup(
name='run-ansible-pull',
version='1.0',
description='Runs Ansible Pull and reports to Sensu',
author='Author',
author_email='[email protected]',
packages=['singleton_test'],
entry_points = {
'console_scripts': [
'singleton-test = singleton_test:main',
],
},
install_requires=[
'tendo==0.2.8',
],
)
# singleton_test/__init__.py
from tendo import singleton
me = singleton.SingleInstance()
def main():
print "Hello, World!"
And here is the full transcript:
$ virtualenv .venv && source .venv/bin/activate
$ python setup.py install && singleton-test
running install
running bdist_egg
running egg_info
creating run_ansible_pull.egg-info
writing requirements to run_ansible_pull.egg-info/requires.txt
writing run_ansible_pull.egg-info/PKG-INFO
writing top-level names to run_ansible_pull.egg-info/top_level.txt
writing dependency_links to run_ansible_pull.egg-info/dependency_links.txt
writing entry points to run_ansible_pull.egg-info/entry_points.txt
writing manifest file 'run_ansible_pull.egg-info/SOURCES.txt'
reading manifest file 'run_ansible_pull.egg-info/SOURCES.txt'
writing manifest file 'run_ansible_pull.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.10-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/singleton_test
copying singleton_test/__init__.py -> build/lib/singleton_test
creating build/bdist.macosx-10.10-x86_64
creating build/bdist.macosx-10.10-x86_64/egg
creating build/bdist.macosx-10.10-x86_64/egg/singleton_test
copying build/lib/singleton_test/__init__.py -> build/bdist.macosx-10.10-x86_64/egg/singleton_test
byte-compiling build/bdist.macosx-10.10-x86_64/egg/singleton_test/__init__.py to __init__.pyc
creating build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
copying run_ansible_pull.egg-info/PKG-INFO -> build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
copying run_ansible_pull.egg-info/SOURCES.txt -> build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
copying run_ansible_pull.egg-info/dependency_links.txt -> build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
copying run_ansible_pull.egg-info/entry_points.txt -> build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
copying run_ansible_pull.egg-info/requires.txt -> build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
copying run_ansible_pull.egg-info/top_level.txt -> build/bdist.macosx-10.10-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/run_ansible_pull-1.0-py2.7.egg' and adding 'build/bdist.macosx-10.10-x86_64/egg' to it
removing 'build/bdist.macosx-10.10-x86_64/egg' (and everything under it)
Processing run_ansible_pull-1.0-py2.7.egg
Removing /private/tmp/singleton-test/.venv/lib/python2.7/site-packages/run_ansible_pull-1.0-py2.7.egg
Copying run_ansible_pull-1.0-py2.7.egg to /private/tmp/singleton-test/.venv/lib/python2.7/site-packages
run-ansible-pull 1.0 is already the active version in easy-install.pth
Installing singleton-test script to /private/tmp/singleton-test/.venv/bin
Installed /private/tmp/singleton-test/.venv/lib/python2.7/site-packages/run_ansible_pull-1.0-py2.7.egg
Processing dependencies for run-ansible-pull==1.0
Searching for tendo==0.2.8
Best match: tendo 0.2.8
Processing tendo-0.2.8-py2.7.egg
tendo 0.2.8 is already the active version in easy-install.pth
Using /private/tmp/singleton-test/.venv/lib/python2.7/site-packages/tendo-0.2.8-py2.7.egg
Finished processing dependencies for run-ansible-pull==1.0
Hello, World!
Exception ImportError: 'No module named sys' in <bound method SingleInstance.__del__ of <tendo.singleton.SingleInstance instance at 0x108618488>> ignored
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Using
from tendo import singleton
singleton.SingleInstance() # will sys.exit(-1) if other instance is running
We get
Exception ignored in: <bound method SingleInstance.__del__ of <tendo.singleton.SingleInstance object at 0x7faf9af8bf60>>
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/tendo/singleton.py", line 63, in __del__
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2222, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 2155, in _find_spec
TypeError: 'NoneType' object is not iterable
2.26user 0.51system 1:31.64elapsed 3%CPU (0avgtext+0avgdata 27284maxresident)k
18944inputs+936outputs (61major+9800minor)pagefaults 0swaps
Proposed fix:
Remove these 2 lines:
import sys
import os
from
def __del__(self):
import sys
import os
if not self.initialized:
return
try:
if sys.platform == 'win32':
if hasattr(self, 'fd'):
os.close(self.fd)
os.unlink(self.lockfile)
else:
import fcntl
fcntl.lockf(self.fp, fcntl.LOCK_UN)
# os.close(self.fp)
if os.path.isfile(self.lockfile):
os.unlink(self.lockfile)
except Exception as e:
if logger:
logger.warning(e)
else:
print("Unloggable error: %s" % e)
sys.exit(-1)
try:
me = singleton.SingleInstance()
except tendo.singleton.SingleInstanceException:
sys.exit(0)
For some reason, this code produces an error when exiting the program.
Exception ignored in: <function SingleInstance.__del__ at 0x00000224E7351510>
Traceback (most recent call last):
File "C:\Users\sarah\PycharmProjects\stella_manager_gui\venv\lib\site-packages\tendo\singleton.py", line 91, in __del__
AttributeError: 'NoneType' object has no attribute 'exit'
Unloggable error: 'NoneType' object has no attribute 'platform'
I'm not sure how this happens, looking at the source it seems like sys is somehow a NoneType. If it helps, I'm using pyqt5 with qasync together with tendo on Windows 11. This error produces when I close it with the X button on pyqt instead of keyboard interrupt.
The tendo.singleton.SingletonInstance
class should throw an exception so the program can deal with the error instead of being forced to quit.
The user may want some additional logging to be done before exiting.
Executing this script display no output:
#!/usr/bin/env python
print '''
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/www.domektkaczki.pl/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/www.domektkaczki.pl/privkey.pem
Your cert will expire on 2018-06-08. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
"certbot renew"
- If you like Certbot, please consider supporting our work by:
'''
because first line is empty.
Hey folks,
I was trying to use SingleInstance
and thought 'this would be great to use as a contextmanager` ... was about to PR that, then saw the latest on main has it.
Can there be a new release to pypi that includes those changes? Thanks!
Perhaps harmless.
mySystem:myappdir myuser$ nosetests-3.4
SingleInstance lockfile: /var/folders/8m/6dsl12z52g7b_9f4kh1yb0s40000gn/T/-usr-local-bin-nosetests-3-.lock
.
----------------------------------------------------------------------
Ran 1 test in 0.008s
OK
Exception ignored in: <bound method SingleInstance.__del__ of <tendo.singleton.SingleInstance object at 0x104af9c18>>
Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/tendo/singleton.py", line 63, in __del__
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable
mySystem:myappdir myuser$
Or when commenting out the two imports suggested by @efairon in #12 :
mySystem:myappdir myuser$ nosetests-3.4
SingleInstance lockfile: /var/folders/8m/6dsl12z52g7b_9f4kh1yb0s40000gn/T/-usr-local-bin-nosetests-3-.lock
.
----------------------------------------------------------------------
Ran 1 test in 0.009s
OK
'NoneType' object is not iterable
Exception ignored in: <bound method SingleInstance.__del__ of <tendo.singleton.SingleInstance object at 0x102f7cbe0>>
Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/tendo/singleton.py", line 83, in __del__
SystemExit: -1
mySystem:myappdir myuser$
Running my app seems to work, with tendo keeping other instances from starting, but that ignored exception shows up again when I exit using ctrl-c
mySystem:myappdir myuser$ python3 myappdir/myapp.py
SingleInstance lockfile: /var/folders/8m/6dsl12z52g7b_9f4kh1yb0s40000gn/T/-Users-myuser-factory-Py-myapp-myapp-myapp-.lock
please Control-C to exit myapp
RUNNING myapp
^CCLOSING myapp
Exception ignored in: <bound method SingleInstance.__del__ of <tendo.singleton.SingleInstance object at 0x10ced8128>>
Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/tendo/singleton.py", line 63, in __del__
ImportError: import of 'sys' halted; None in sys.modules
mySystem:myappdir myuser$
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Hi,
the singleton.py code contains a race condition:
fcntl.lockf(self.fp, fcntl.LOCK_UN)
if os.path.isfile(self.lockfile):
os.unlink(self.lockfile)
So the lock is released, then the file is deleted, but at that moment the file may already belong to another process.
I tried different methods of unlocking/deleting the file, but they all result in some race condition.
The only reliable way I found is to keep the lock file, and not delete it.
Regards
Dirk
import threading
import os
import time
def fun():
os.system("python test2.py")
for x in range(0,10):
time.sleep(0.2)
for i in range(0,200):
t = threading.Thread(target=fun)
t.start()
import random
from singleton import Singleton
si = Singelton("test",20)
id = random.randint(1,100000)
for i in range(0,50):
f=open("/tmp/testlock","a" )
f.write("%s\n" % id)
f.flush
f.close()
import sys
import os
import time
import fcntl
class Singleton():
def __init__(self,name,timeout):
self.locked = False
self.lockfile = "/tmp/lock_%s" % name
self.fp = open(self.lockfile , 'w')
while timeout > 0:
try:
fcntl.lockf(self.fp, fcntl.LOCK_EX | fcntl.LOCK_NB)
self.locked = True
break
except:
time.sleep(0.2)
timeout = timeout - 0.2
if self.locked == False:
raise Exception("Could not aquire lock %s within %s seconds" % (name,timeout))
def __del__(self):
if self.locked == True:
import os
# all of these fail:
#os.unlink(self.lockfile)
#os.remove(self.lockfile)
#os.system("rm %s" % self.lockfile)
import fcntl
fcntl.lockf(self.fp, fcntl.LOCK_UN) # the reliable way, dont delete !
#if os.path.isfile(self.lockfile):
# os.unlink(self.lockfile)
"cat /tmp/testlock |uniq -c|grep -v 50" should show no lines if test2.py was actually run as a single instance.
windows
xxx = 'python_with_singlton.py'
pros = popen(xxx)
pros.kill()
pros = popen(xxx) - # PermissionError: [WinError 32] The process cannot access the file because it is being used ....
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.