alba-synchrotron/pytango-db

A python implementation of the Tango Database device server

Python Tango Database DS

A python implementation of the Tango Database device server using PyTango.

The Tango interface is the same as the official C++ Tango Database device server.

It comes with a default backend which uses sqlite3.

Installation

$ pip install pytango-db

Start server

$ DataBaseds --db_access=sqlite3 --port=10000 2

That's all folks!

pytango-db's Issues

sqlite3 backend doesn't need threadpool

Since the DatabaseDS is running in gevent mode, all backend requests are already being handled in the same thread. This eliminates the need of sqlite3 having a dedicated threadpool

Problem on Windows seems to be related to gevent

After the following installation:

pip install pytango-db
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting pytango-db
  Downloading https://files.pythonhosted.org/packages/52/11/25aa7e9b69c4e3ece42198085d0733cdf86e48c954ed1cba6b42f63b91aa/pytango-db-0.2.0.tar.gz
Requirement already satisfied: PyTango in c:\lib\site-packages (from pytango-db) (9.2.3)
Collecting gevent (from pytango-db)
  Downloading https://files.pythonhosted.org/packages/5b/16/34bb535206f9eee9ab87647575b7704dba23aca6201a5a1739dcd036a1ca/gevent-1.4.0-cp27-cp27m-win_amd64.whl (3.0MB)
Collecting futures (from pytango-db)
  Downloading https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Requirement already satisfied: six in c:\miniconda-x64\envs\py2qt5\lib\site-packages (from PyTango->pytango-db) (1.12.0)
Collecting cffi>=1.11.5; sys_platform == "win32" and platform_python_implementation == "CPython" (from gevent->pytango-db)
  Downloading https://files.pythonhosted.org/packages/fa/24/37ec704b98ffc36e7d0ade9e4653539c0b8e6fec34f084f30194609aa10b/cffi-1.12.3-cp27-cp27m-win_amd64.whl (168kB)
Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" (from gevent->pytango-db)
  Downloading https://files.pythonhosted.org/packages/94/2e/36d4aa9fc190695f67b5d4918f72f2c5beca7edd3b52bc496973954bcc5d/greenlet-0.4.15-cp27-cp27m-win_amd64.whl
Collecting pycparser (from cffi>=1.11.5; sys_platform == "win32" and platform_python_implementation == "CPython"->gevent->pytango-db)
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
Building wheels for collected packages: pytango-db, pycparser
  Building wheel for pytango-db (setup.py): started
  Building wheel for pytango-db (setup.py): finished with status 'done'
  Stored in directory: C:\Users\appveyor\AppData\Local\pip\Cache\wheels\fa\f8\49\f679f6a4278e07acb0a069bd02fecdea37100a67854e1e4b87
  Building wheel for pycparser (setup.py): started
  Building wheel for pycparser (setup.py): finished with status 'done'
  Stored in directory: C:\Users\appveyor\AppData\Local\pip\Cache\wheels\f2\9a\90\de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
Successfully built pytango-db pycparser
Installing collected packages: pycparser, cffi, greenlet, gevent, futures, pytango-db
Successfully installed cffi-1.12.3 futures-3.2.0 gevent-1.4.0 greenlet-0.4.15 pycparser-2.19 pytango-db-0.2.0

I tried this on Windows and have the following error:

Exiting: Server exited with tango.DevFailed:
DevFailed[
DevError[
    desc = AttributeError: 'module' object has no attribute 'Event'
           
  origin =   File "c:\miniconda-x64\envs\py2qt5\lib\site-packages\tangodb\database.py", line 1820, in device_factory
    device = self._new_device(deviceImplClass, klass, dev_name)
  File "C:\Lib\site-packages\tango\device_class.py", line 547, in __DeviceClass__new_device
    return klass(dev_class, dev_name)
  File "C:\Lib\site-packages\tango\server.py", line 571, in __init__
    self.init_device()
  File "C:\Lib\site-packages\tango\server.py", line 360, in init_device
    return get_worker().execute(init_device_orig, self)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 183, in execute
    task = self.submit(fn, *args, **kwargs)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 177, in submit
    return self.loop.submit(fn, *args, **kwargs)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 145, in submit
    event = gevent._threading.Event()
  reason = PyDs_PythonError
severity = ERR]
]
Traceback (most recent call last):
  File "C:\Lib\site-packages\tango\server.py", line 1479, in run
    return server_run()
  File "C:\Lib\site-packages\tango\server.py", line 1335, in __server_run
    worker.run(tango_loop, wait=True)
  File "C:\Lib\site-packages\tango\green.py", line 101, in run
    return self.access(accessor, timeout=timeout)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 174, in access
    return accessor.get(timeout=timeout)
  File "src\gevent\event.py", line 268, in gevent._event.AsyncResult.get
  File "src\gevent\event.py", line 296, in gevent._event.AsyncResult.get
  File "src\gevent\event.py", line 286, in gevent._event.AsyncResult.get
  File "src\gevent\event.py", line 266, in gevent._event.AsyncResult._raise_exception
  File "c:\miniconda-x64\envs\py2qt5\lib\site-packages\gevent\threadpool.py", line 281, in _worker
    value = func(*args, **kwargs)
  File "C:\Lib\site-packages\tango\server.py", line 1329, in tango_loop
    util.server_init()
DevFailed: DevFailed[
DevError[
    desc = AttributeError: 'module' object has no attribute 'Event'
           
  origin =   File "c:\miniconda-x64\envs\py2qt5\lib\site-packages\tangodb\database.py", line 1820, in device_factory
    device = self._new_device(deviceImplClass, klass, dev_name)
  File "C:\Lib\site-packages\tango\device_class.py", line 547, in __DeviceClass__new_device
    return klass(dev_class, dev_name)
  File "C:\Lib\site-packages\tango\server.py", line 571, in __init__
    self.init_device()
  File "C:\Lib\site-packages\tango\server.py", line 360, in init_device
    return get_worker().execute(init_device_orig, self)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 183, in execute
    task = self.submit(fn, *args, **kwargs)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 177, in submit
    return self.loop.submit(fn, *args, **kwargs)
  File "C:\Lib\site-packages\tango\gevent_executor.py", line 145, in submit
    event = gevent._threading.Event()
  reason = PyDs_PythonError
severity = ERR]
]
Exited

It's strange since on Linux it works with the same version of gevent without problems:

:~> python -c "import gevent; print(gevent.__version__)"
1.4.0
:~> DataBaseds --db_access=sqlite3 --port=20000 2
Ready to accept request

Any clue?

segfault in put_device_property

I use the following code:

In [1]: import tango

In [2]: db = tango.Database()

In [3]: db.put_device_property("sys/tg_test/1", {"test_prop":1})

and get:

Core was generated by `/usr/bin/python /home/zreszela/.local/bin/DataBaseds --db                                                                              _access=sqlite3 --port'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __strncpy_sse2_unaligned ()
    at ../sysdeps/x86_64/multiarch/strcpy-sse2-unaligned.S:296
296     ../sysdeps/x86_64/multiarch/strcpy-sse2-unaligned.S: No such file or dir                                                                              ectory.
[Current thread is 1 (Thread 0x7fb38201b700 (LWP 6294))]
(gdb) bt
#0  __strncpy_sse2_unaligned ()
    at ../sysdeps/x86_64/multiarch/strcpy-sse2-unaligned.S:296
#1  0x00007fb39d7fcd9f in strncpy (__len=18446744073709551615,
    __src=<optimized out>, __dest=0x7fb3740053d0 "x\003")
    at /usr/include/x86_64-linux-gnu/bits/string3.h:126
#2  from_str_to_char (in=in@entry=0)
    at /home/zreszela/workspace/pytango/ext/pyutils.cpp:104
#3  0x00007fb39d7d08f5 in PyString_AsCorbaString (obj_ptr=obj_ptr@entry=0)
    at /home/zreszela/workspace/pytango/ext/from_py.cpp:24
#4  0x00007fb39d890727 in from_py<8l>::convert (
    tg=<synthetic pointer>: <optimized out>, o=0)
    at /home/zreszela/workspace/pytango/ext/fast_from_py.h:186
#5  fast_python_to_corba_buffer_sequence<16l> (
    res_dim_x=<synthetic pointer>: <optimized out>, fname="insert_array",
    pdim_x=0x0, py_val=['CtrlSystem', '1', 'Services', 0])
    at /home/zreszela/workspace/pytango/ext/fast_from_py.h:462
#6  fast_python_to_corba_buffer_numpy<16l> (
    res_dim_x=<synthetic pointer>: <optimized out>, fname="insert_array",
    pdim_x=0x0, py_val=<optimized out>)
    at /home/zreszela/workspace/pytango/ext/fast_from_py_numpy.hpp:251
#7  fast_convert2array<16l> (o=...)
    at /home/zreszela/workspace/pytango/ext/fast_from_py.h:508
#8  insert_array<16l> (o=..., any=...)
---Type <return> to continue, or q <return> to quit---
   erver/command.cpp:169
#9  0x00007fb39d88a175 in PyCmd::execute (this=0x7fb378012a10, dev=<optimized out>, param_any=...)
    at /home/zreszela/workspace/pytango/ext/server/command.cpp:339
#10 0x00007fb39cd5a4e4 in Tango::DeviceClass::command_handler (this=0x7fb37800b550, device=device@entry=0x7fb378018500, command="DbGetProperty", in_any=...)
    at deviceclass.cpp:1199
#11 0x00007fb39cd0fbb8 in Tango::DeviceImpl::command_inout (this=0x7fb378018500, in_cmd=0x7fb374002e10 "DbGetProperty", in_any=...) at device.cpp:1442
#12 0x00007fb39cd2ca66 in Tango::Device_2Impl::command_inout_2 (this=0x7fb378018500, in_cmd=<optimized out>, in_data=..., source=Tango::CACHE_DEV)
    at device_2.cpp:440
#13 0x00007fb39cd4638a in Tango::Device_4Impl::command_inout_4 (this=0x7fb378018500, in_cmd=0x7fb374002e10 "DbGetProperty", in_data=...,
    source=Tango::CACHE_DEV, cl_id=...) at device_4.cpp:475
#14 0x00007fb39ceeefe2 in _0RL_lcfn_6fe2f94a21a10053_a3000000 (cd=0x7fb38201a9c0, svnt=<optimized out>) at tangoSK.cpp:5196
#15 0x00007fb39b5b6b69 in omniCallHandle::upcall(omniServant*, omniCallDescriptor&) () from /usr/lib/libomniORB4.so.1
#16 0x00007fb39cf088dc in Tango::_impl_Device_4::_dispatch (this=<optimized out>, _handle=...) at tangoSK.cpp:5750
#17 0x00007fb39cf0996b in Tango::_impl_Device_5::_dispatch (this=<optimized out>, _handle=...) at tangoSK.cpp:7220
#18 0x00007fb39b5aff6d in omni::omniOrbPOA::dispatch(omniCallHandle&, omniLocalIdentity*) () from /usr/lib/libomniORB4.so.1
#19 0x00007fb39b58f116 in omniLocalIdentity::dispatch(omniCallHandle&) () from /usr/lib/libomniORB4.so.1
#20 0x00007fb39b5cfd20 in omni::GIOP_S::handleRequest() () from /usr/lib/libomniORB4.so.1
#21 0x00007fb39b5d0a98 in omni::GIOP_S::dispatcher() () from /usr/lib/libomniORB4.so.1
#22 0x00007fb39b5cd6c5 in omni::giopWorker::real_execute() () from /usr/lib/libomniORB4.so.1
#23 0x00007fb39b5cdd4f in omni::giopWorker::execute() () from /usr/lib/libomniORB4.so.1
#24 0x00007fb39b58402d in omniAsyncWorkerInfo::run() () from /usr/lib/libomniORB4.so.1
#25 0x00007fb39cec3c4c in Tango::create_PyPerThData (info=...) at utils.cpp:3234
#26 0x00007fb39b583f24 in omniAsyncWorkerInfo::run() () from /usr/lib/libomniORB4.so.1
#27 0x00007fb39b5845ff in omniAsyncWorker::run(void*) () from /usr/lib/libomniORB4.so.1
#28 0x00007fb39b2c56d9 in omni_thread_wrapper () from /usr/lib/libomnithread.so.3
#29 0x00007fb39ec3b494 in start_thread (arg=0x7fb38201b700) at pthread_create.c:333
#30 0x00007fb39e058acf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

My PyTango version is 9.3.0.

Create entry points for other extensions

Need to create entry point so other plugins (ex: bliss) can be installed easily

State of this code?

From tango-controls/pytango#266 it sounded like this should be a drop-in replacement for the database. However, a very basic use (start server with empty database, open Jive) gives me

TypeError: expected string or Unicode object, int found for DbGetProperty.

It seems that here should be str(n_rows). After fixing this, the new error is

AttributeError: 'Sqlite3Database' object has no attribute 'get_server_name_list'

This looks like a pretty basic unimplemented method?