ldo / dbussy Goto Github PK

View Code? Open in Web Editor NEW

89.0 8.0 22.0 568 KB

Python binding for D-Bus using asyncio

d-bus dbus asyncio coroutines ctypes ctypes-wrapper libdbus

dbussy's People

Contributors

Stargazers

Watchers

dbussy's Issues

How to strip signatures from arguments

Hello,
I am sure this issue has already been raised, but I could not find an existing solution
Is there a way to strip all the signatures from arguments received from ravel?
I need this to convert a python dbus program to ravel
Thank you in advance for you reply

Example of ravel.ManagedObjectsHandler().get_managed_objects(...)

Hello,
I would like to replace python dbus by dbussy/ravel
I think I could use ravel.ManagedObjectsHandler().get_managed_objects(...) to get the managed objects of / at org.bluez
I would however not know how to do this, because I do not understand all the variables of the method
I would therefore be very grateful if you could provide an example of how this is done
Thank you in advance

Low Performance for Fixed Type Arrays

Performance of sending and receiving DBus messages with arrays of fixed type is low. We implemented DBus services, that transmit messages which contain ay arrays. Even for rather short arrays of 10000 elements, we observed runtimes of several seconds for appending and extracting data (see AppendIter and ExtractIter in dbussy.py). The reason is the generic implementation, which creates an iterator object for every single byte, that is append to or extracted from the DBus data.
@ldo: I solved this problem using the unused implementations AppendIter.append_fixed_array() and ExtractIter.fixed_array in a github fork.
I would appreciate a review.

ravel.signal objects_added always has '/' as path

I have:

self._bus.listen_objects_added(func=self.objects_added)
...
    @ravel.signal(name="object_added",
                  in_signature="oa{sa{sv}}",
                  path_keyword="object_path",
                  args_keyword="args",)
    def objects_added(self, object_path, args):
        logging.warning(object_path)
        logging.warning(args)

But object path always is '/' where I, for example, should expect '/org/bluez/hci0/dev_09_6B_A1_1D_B3_E3':

22-05-2019 16:23:27.760 WARNING [aioble.objects_added:90] /
22-05-2019 16:23:27.760 WARNING [aioble.objects_added:91] [ObjectPath('/org/bluez/hci0/dev_09_6B_A1_1D_B3_E3'), {'org.bluez.Device1': {'ServicesResolved': (Signature('b'), False), 'Trusted': (Signature('b'), False), 'UUIDs': (Signature('as'), []), 'LegacyPairing': (Signature('b'), False), 'Paired': (Signature('b'), False), 'Adapter': (Signature('o'), ObjectPath('/org/bluez/hci0')), 'Blocked': (Signature('b'), False), 'Connected': (Signature('b'), False), 'Address': (Signature('s'), '09:6B:A1:1D:B3:E3'), 'Alias': (Signature('s'), '09-6B-A1-1D-B3-E3'), 'ManufacturerData': (Signature('a{qv}'), {6: (Signature('ay'), [1, 9, 32, 2, 225, 87, 110, 71, 58, 147, 111, 255, 164, 202, 51, 230, 108, 180, 229, 6, 238, 133, 47, 2, 236, 79, 249])}), 'RSSI': (Signature('n'), -49)}, 'org.freedesktop.DBus.Properties': {}, 'org.freedesktop.DBus.Introspectable': {}}]

Perhaps I don't understand it completely yet.

Diagnosis: Name Issue

unregister specific interface

When trying to unregister a specific interface from a path the unregister fails.

I think the problem may be here:
Line 597:
If interface != None then the list is a list of _Interface objects while if interface == None it's a list of interface names (strings).

Steep learning curve

Hi,
first and foremost: Thanks for all the good work.

I'm trying to use dbussy/ravel to interface ConnMan.
Since I'm new to all of them: ConnMan, D-Bus and Your code, there's little surprise in me having problems, sorry.

I am trying to do things stepwise and I have the following test code:

#!/usr/bin/python3

import asyncio
import ravel

import pprint
from logging import getLogger
log = getLogger('networks')
pp = pprint.PrettyPrinter(indent=4)

bus_name = "net.connman"
path_name = "/"
iface_name = "net.connman.Manager"
chime_signal_name = "chime"

loop = asyncio.get_event_loop()


bus = ravel.system_bus()
bus.attach_asyncio(loop)
server = loop.run_until_complete(bus.get_proxy_interface_async(destination=bus_name,
                                                               path="/",
                                                               interface=iface_name))
server = server(connection=bus.connection, dest=bus_name)


async def main():
    print('===================================')
    tl = await server['/'].GetTechnologies()
    for p in tl[0]:
        # pp.pprint(p)
        print('-----------------------------------')
        print(f'-- {p[1]["Name"][1]}: {p[0]} --')
    print('===================================')
    sl = await server['/'].GetServices()
    for p in sl[0]:
        # pp.pprint(p)
        print('-----------------------------------')
        print(f'-- {p[1]["Name"][1]}: {p[0]} --')
    print('===================================')
    for p in tl[0]:
        pp.pprint(p)
        if p[1]['Type'][1] == 'wifi':
            print('-----------------------------------')
            t = p[1]
            print(f'-- {t["Name"][1]}: {p[0]} --')
            s = bus.get_proxy_interface(bus_name,  p[0], 'net.connman.Technology')
            s(connection=bus.connection, dest='net.connman.Technology').Scan()
            await asyncio.sleep(10)
    print('===================================')
    sl = await server['/'].GetServices()
    for p in sl[0]:
        # pp.pprint(p)
        print('-----------------------------------')
        print(f'-- {p[1]["Name"][1]}: {p[0]} --')
    print('===================================')


loop.run_until_complete(main())

This code, mostly stealed from useless_managed_object_server_ravelled, results in an exception:

===================================
-----------------------------------
-- Wired: /net/connman/technology/ethernet --
-----------------------------------
-- WiFi: /net/connman/technology/wifi --
-----------------------------------
-- Bluetooth: /net/connman/technology/bluetooth --
===================================
-----------------------------------
-- NETGEAR_11AC: /net/connman/service/wifi_001500d77076_4e4554474541525f31314143_managed_psk --
-----------------------------------
-- Wired: /net/connman/service/ethernet_02693694c027_cable --
-----------------------------------
-- Wired: /net/connman/service/ethernet_723575276940_cable --
-----------------------------------
-- Wired: /net/connman/service/ethernet_ae8d48867953_cable --
-----------------------------------
-- Wired: /net/connman/service/ethernet_ba6926867953_cable --
-----------------------------------
-- Wired: /net/connman/service/ethernet_366cdd276940_cable --
===================================
[   ObjectPath('/net/connman/technology/ethernet'),
    {   'Connected': (Signature('b'), True),
        'Name': (Signature('s'), 'Wired'),
        'Powered': (Signature('b'), True),
        'Tethering': (Signature('b'), False),
        'Type': (Signature('s'), 'ethernet')}]
[   ObjectPath('/net/connman/technology/wifi'),
    {   'Connected': (Signature('b'), True),
        'Name': (Signature('s'), 'WiFi'),
        'Powered': (Signature('b'), True),
        'Tethering': (Signature('b'), False),
        'Type': (Signature('s'), 'wifi')}]
-----------------------------------
-- WiFi: /net/connman/technology/wifi --
Traceback (most recent call last):
  File "/home/mcon/projects/dbussy/networks.py", line 78, in <module>
    loop.run_until_complete(main())
  File "/usr/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/home/mcon/projects/dbussy/networks.py", line 67, in main
    s(connection=bus.connection, dest='net.connman.Technology').Scan()
AttributeError: 'net.connman.Technology_factory' object has no attribute 'Scan'

Process finished with exit code 1

Reference code I'm trying to convert to ravel is:

    technology = dbus.Interface(bus.get_object("net.connman", 
                                               "/net/connman/technology/wifi"),
                                "net.connman.Technology")
    technology.Scan()

What am I doing so wrong?
Thanks in Advance
Mauro

Publish the source to PYPI

Hi,
currently pypi has only the wheel version of the package available.
Can you publish also the source version?
This is useful in some case (like yocto) were we cannot use the wheel directly
Thanks
Nicola Lunghi

Unable to duplicate bus filter

Hello,

First off thank you so much for sharing this library! I'm working on an asyncio bluetooth low energy (bluez) library utilizing your library.

I have a class that sets up a callback filter using add_filter & bus_add_match. My issue is that when multiple instances of this class are created, only the first instance to register the filter is receiving all the message signals. I have been sure to include the path, since that is the only differentiator between the bus_add_match calls. My question therefore is, should I be treating the bus_add_match as a global rule, or should they be able to register match rules individually?

Hopefully this is clear enough, happy to expand if not.
Thanks!
-Kent

High CPU usage for `_reaper`

Hey!

After profiling a program, I have noticed that the _reaper function is called very often and responsible for some quite CPU usage due to this. From my understanding, it is running at each tick as long as there is a DBus related task running, which is always if you are listening for signals.

I would suggest to just sleep a bit inside reaper.

  5002518   18.592    0.000  752.277    0.000 base_events.py:1815(_run_once)
  5002518    5.646    0.000  681.874    0.000 selectors.py:452(select)
  5002518  673.818    0.000  673.818    0.000 {method 'poll' of 'select.epoll' objects}
  5007678    3.841    0.000   45.105    0.000 events.py:78(_run)
  5007678    2.863    0.000   41.264    0.000 {method 'run' of 'Context' objects}
  5000867   11.825    0.000   34.350    0.000 dbussy.py:1454(_reaper)
  5003419    4.501    0.000   20.014    0.000 base_events.py:736(call_soon)
  5003419    5.591    0.000   14.493    0.000 base_events.py:765(_call_soon)
  5007620    5.966    0.000    8.111    0.000 events.py:31(__init__)

Formal Release?

Do you consider this package ready for prime time? If so, could you tag a commit with a release version?

A signed commit would be great, but not required. A signed tar would be great as well (example).

Typo in Connection.unregister

There are two typos in Connection.unregister, raising an exception when called. See here.

Help needed to migrating a wifi agent

Hello @ldo

As you probably know, I am migrating code from python dbus to ravel
I have already migrated methods and signals

But I am stuck at the wifi agent, see the code below
I am not sure how to write the class and method decorators and what the functions should return
I tried to follow the examples, without success

Would you please be so kind to help me out with:

the decorator for the class
the decorator for the RequestInput method
and what the RequestInput method should return

Thank you in advance for your assistance

class connmanWifiAgent(dbus.service.Object):

    @dbus.service.method('net.connman.Agent', in_signature='oa{sv}', out_signature='a{sv}')
    def RequestInput(self, path, fields):

    @dbus.service.method('net.connman.Agent', in_signature='', out_signature='')
    def Release(self):

    @dbus.service.method('net.connman.Agent', in_signature='os', out_signature='')
    def RequestBrowser(self, path, url):

    @dbus.service.method('net.connman.Agent', in_signature='os', out_signature='')
    def ReportError(self, path, error):

    @dbus.service.method('net.connman.Agent', in_signature='', out_signature='')
    def Cancel(self):

Make this package available via pip?

The ability to use the async functionality in Python 3 is very useful as you describe.

It would be nice to pull the package via pip3 instead of having to clone and install manually. Will you be doing this?

Cannot run loop in IDE

Thank you for this project! Just starting out, I can run the introspect example fine from the linux terminal but not inside my IDE (Spyder3)

runfile('/home/graham/GitProjects/dbussy_examples/introspect', args='system org.bluez /org/bluez', wdir='/home/graham/GitProjects/dbussy_examples')
Traceback (most recent call last):

  File "<ipython-input-1-9cb732248004>", line 1, in <module>
    runfile('/home/graham/GitProjects/dbussy_examples/introspect', args='system org.bluez /org/bluez', wdir='/home/graham/GitProjects/dbussy_examples')

  File "/usr/lib/python3/dist-packages/spyder_kernels/customize/spydercustomize.py", line 827, in runfile
    execfile(filename, namespace)

  File "/usr/lib/python3/dist-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/graham/GitProjects/dbussy_examples/introspect", line 47, in <module>
    reply = conn.loop.run_until_complete(await_reply)

  File "/usr/lib/python3.8/asyncio/base_events.py", line 592, in run_until_complete
    self._check_running()

  File "/usr/lib/python3.8/asyncio/base_events.py", line 552, in _check_running
    raise RuntimeError('This event loop is already running')

RuntimeError: This event loop is already running

Is there a workaround?

How to pass signatures

Hello again
I trying to find out how to use ravel to call the SetProperty method below with name='Name' and value=['A', 'B']

<method name="SetProperty">
    <arg name="name" type="s" direction="in"/>
    <arg name="value" type="v" direction="in"/>
</method>

Thank you for your help

Possible examples

Hello again @ldo

As promised, here is what works for me to migrate legacy code from python dbus to ravel

This might be useful to someone else and maybe find a place in dbussy_examples in a form or another
I am also interested in feedback, because I am new to asyncio/dbus/connman

Move away from distutils

As of Python 3.10 distutils is deprecated and will be removed in 3.12:

https://docs.python.org/3/whatsnew/3.10.html#distutils-deprecated

Migrating to setuptools will be fairly trivial.

Unreleased ref to pending_done in PendingCall

Hello,
First of all thank you for sharing and maintaining this library.
I'm using dbussy to obtain login1.Manager Inhibit lock as follows:

async def take(conn, inh, what, how):
    message = dbus.Message.new_method_call(
        destination="org.freedesktop.login1",
        path="/org/freedesktop/login1",
        iface="org.freedesktop.login1.Manager",
        method="Inhibit")
    argobjs = [what, 'My Test', 'Testing', how]
    sig = [dbus.BasicType(dbus.TYPE.STRING)] * 4
    message.append_objects(sig, *argobjs)
    reply = await conn.send_await_reply(message)
    inh.fd = int(reply.expect_return_objects("h")[0])

class Inhibitor(object):
    def __init__(self):
        self.fd = -1

inh = Inhibitor()
loop = asyncio.get_event_loop()
conn = dbus.Connection.bus_get(DBUS.BUS_SYSTEM, private=False)
conn.attach_asyncio(loop)
task = loop.create_task(take(conn, inh, 'sleep', 'delay'))
loop.run_forever()

Getting file descriptor from reply will return duplicated FD and closing returned FD will release Inhibit lock.
From documentations of dbus_message_iter_append_basic():

For Unix file descriptors this function will internally duplicate the descriptor you passed in. Hence you may close the descriptor immediately after this call.

However original FD not closed since PendingCall.await_reply..pending_done not released.
python3 14154 root 7w FIFO 0,22 0t0 225 /run/systemd/inhibit/16.ref
python3 14154 root 8w FIFO 0,22 0t0 225 /run/systemd/inhibit/16.ref

Checking with gc return the following references:

<dbussy.PendingCall object at 0x7f7acac99570>
<function PendingCall.await_reply.<locals>.pending_done at 0x7f7acacad158>
<weakref at 0x7f7acaca03b8; dead>
<cell at 0x7f7acacfdd68: function object at 0x7f7acacad158>
<cell at 0x7f7acacfdd98: PendingCall object at 0x7f7acac99570>
<cell at 0x7f7acacfddc8: weakref object at 0x7f7acaca03b8>
<function PendingCall.set_notify.<locals>._wrap_notify at 0x7f7acacad0d0>
<_ctypes.CThunkObject object at 0x7f7acaca8430>
(<cell at 0x7f7acacfdd08: _asyncio.Future object at 0x7f7acac95828>,)
(<cell at 0x7f7acacfdd68: function object at 0x7f7acacad158>, <cell at 0x7f7acacfdd98: PendingCall object at 0x7f7acac99570>, <cell at 0x7f7acacfddc8: weakref object at 0x7f7acaca03b8>)
<cell at 0x7f7acacfdd08: _asyncio.Future object at 0x7f7acac95828>

Proposed fix:

diff --git a/dbussy.py b/dbussy.py
index c947b59..e4934db 100644
--- a/dbussy.py
+++ b/dbussy.py
@@ -4918,6 +4918,7 @@ class PendingCall :
         self.set_notify(pending_done, weak_ref(self))
           # avoid reference circularity self → pending_done → self
         reply = await done
+        self.set_notify(None, None)
         return \
             reply
     #end await_reply

ctypes.c_ulong size on arm

The size of c_long on arm differs from x86_64.

ctypes.sizeof(ctypes.c_long)  # returns 4 on arm
ctypes.sizeof(ctypes.c_long)  # returns 8 on x86_64
ctypes.sizeof(ctypes.c_longlong)  # returns 8 on arm and x86_64

Using c_long results with wrong primitive type conversion for DBus 'X' and 'T'.
It would be safe to use c_longlong (c_ulonglong) instead as they are 8 bytes on both architectures.

--- a/dbussy.py 2019-03-13 13:32:14.099119209 +0100
+++ b/dbussy.py 2019-03-13 13:27:13.787765219 +0100
@@ -76,8 +76,8 @@
             TYPE_UINT16 : ct.c_ushort,
             TYPE_INT32 : ct.c_int,
             TYPE_UINT32 : ct.c_uint,
-            TYPE_INT64 : ct.c_long,
-            TYPE_UINT64 : ct.c_ulong,
+            TYPE_INT64 : ct.c_longlong,
+            TYPE_UINT64 : ct.c_ulonglong,
             TYPE_DOUBLE : ct.c_double,
             TYPE_STRING : ct.c_char_p,
             TYPE_OBJECT_PATH : ct.c_char_p,

Argument name=None raises Exception in ravel.method and ravel.signal

It tries to validate the value, which fails for None.
Maybe try this patch?
Btw: Thanks for the software!

Can not register coroutines with add_filter in asyncio debug mode

Hi,

Starting a script using dbussy with asyncio debug mode makes asynchronous dbus callbacks to stop being called. I tracked the problem down to when the coroutines are called, and it seems to be supported, but it ends up with a traceback.

I have written this patch on dbussy_example bus_monitor to illustrate the problem I face:

diff --git a/bus_monitor b/bus_monitor
index a086df4..fc9cee8 100755
--- a/bus_monitor
+++ b/bus_monitor
@@ -23,7 +23,7 @@ loop = asyncio.get_event_loop()
 # Mainline
 #-
 
-def handle_message(conn, message, _) :
+async def handle_message(conn, message, _) :
     sys.stdout.write \
       (
             "%s: %s.%s[%s](%s)\n"

Then, launching bus_monitor with the asyncio debug mode :

# PYTHONASYNCIODEBUG=1 ./bus_monitor system
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 259, in 'converting callback result'
TypeError: an integer is required (got type CoroWrapper)
Exception ignored in: <function Connection.add_filter.<locals>.wrap_function at 0x7fe25766e268>
<CoroWrapper handle_message() running at ./bus_monitor:26, created at /usr/lib/python3.5/asyncio/coroutines.py:80> was never yielded from
Coroutine object created at (most recent call last):
  File "./bus_monitor", line 55, in <module>
    loop.run_forever()
  File "/usr/lib/python3.5/asyncio/base_events.py", line 345, in run_forever
    self._run_once()
  File "/usr/lib/python3.5/asyncio/base_events.py", line 1304, in _run_once
    handle._run()
  File "/usr/lib/python3.5/asyncio/events.py", line 125, in _run
    self._callback(*self._args)
  File "/usr/local/lib/python3.5/dist-packages/dbussy.py", line 1710, in handle_watch_event
    call_dispatch()
  File "/usr/local/lib/python3.5/dist-packages/dbussy.py", line 1689, in call_dispatch
    status = dispatch()
  File "/usr/local/lib/python3.5/dist-packages/dbussy.py", line 2236, in dispatch
    dbus.dbus_connection_dispatch(self._dbobj)
  File "/usr/local/lib/python3.5/dist-packages/dbussy.py", line 2464, in wrap_function
    result = function(self, Message(dbus.dbus_message_ref(message)), user_data)
  File "/usr/lib/python3.5/asyncio/coroutines.py", line 80, in debug_wrapper
    return CoroWrapper(gen, None)
/usr/local/lib/python3.5/dist-packages/dbussy.py:2236: RuntimeWarning: coroutine 'handle_message' was never awaited
  dbus.dbus_connection_dispatch(self._dbobj)

My understanding of the problem is that the proper way to detect if an object is a coroutine is to use the asyncio function iscoroutine, instead of checking if the object is of CoroutineType.

Something like:

- if isinstance(result, types.CoroutineType) :
+ if asyncio.iscoroutine(result) :

If I replace all the occurences of this isinstance(result, types.CoroutineType) by an asyncio.iscoroutine(result), I don't get the tracebacks anymore.

Ravel add client paths to server space with listen_signal()

I'm using ravel for server functions, with appropriate attach_asyncio() and request_name() calls.

I'm also monitoring NetworkManager DBus signals, using the listen_signal() method on the same bus.

This is causing freedesktop interfaces on e.g. /org/freedesktop/NetworkManager to be populated under the local server bus name.

Is this expected behavior?

Message.del can lead to file descriptor deadlock

tl;dr: Relying on Message.__del__ to call dbus_message_unref() can leave duplicate file descriptors open that can cause deadlock scenarios depending on what the Python garbage collector does. Replacing reliance on __del__ with explicit cleanup of Message objects in Ravel will prevent these deadlocks.

libdbus calls dup() both when a file descriptor is added to a request DBusMessage as an argument and when a file descriptor is retrieved from a reply DBusMessage as a result. Since a DBussy Message wraps a libdbus DBusMessage, this means that every Message object is carrying around duplicates for any file descriptors added to or extracted from it.

DBussy uses the Message.__del__ method to call libdbus' dbus_message_unref(), which closes those duplicate file descriptors.

The combination of these two facts can result in deadlock during scenarios where a DBus client or service is depending on the other side to close a file descriptor but the Python garbage collector hasn't reaped the Message object. An example would be when a client sends the writer FD of a pipe to a DBus service as a means to receive a bulk data transfer. Even if the service Python code closes the file descriptor request argument it receives from DBussy, its end of the pipe will remain open if the dangling Message object hasn't been picked up by the Python garbage collector and its __del__ method invoked.

You can reproduce this as follows:

Run a small Ravel DBus service that exposes an interface that accepts one file descriptor, writes b"Hello" to it, closes it, and returns a success DBus response.
Run a small Ravel DBus client that:
1. Creates a pipe via r,w = os.pipe().
2. Sends the w file descriptor to the DBus service as an request argument.
3. Waits for a successful response from the DBus service.
4. Attempts to read what the service sent, via with open(r, "rb") as f: f.read()
Observe that on some runs the client will block indefinitely waiting for more data from the service's end of the pipe. (Sometimes it may not block, if the Python GC on the service side collects the Message object. In those cases, just run the client again without restarting the service; within 2-3 requests I usually hit the scenario in the service code where the Python GC doesn't reap the Message object.)

This issue is most visible on the service side, but you can also run into it on the client side as well where a Message object isn't collected and its dup() of the file descriptor that was passed in a DBus request is still open.

This issue should be resolvable by replacing Message.__del__ with a cleanup method that must be explicitly invoked when a Message is done with (and updating Ravel to use it to explicitly clean up all Message objects it creates on the user's behalf).

Cannot return a single value

It does not seem to be possible for a method to return a single value. Any method that attempts to do so will raise a ValueError. The specific exception depends on the type.

Here is a short example that implements a method for each type. Each method that returns a single value fails, while the methods that return multiple values work correctly.

% python -m pip show dbussy
Name: DBussy
Version: 1.3
Summary: language bindings for libdbus, for Python 3.5 or later
Home-page: https://github.com/ldo/dbussy
Author: Lawrence D'Oliveiro
Author-email: [email protected]
License: LGPL v2.1+
Location: /home/dhatch/.local/lib/python3.8/site-packages
Requires:
Required-by:

Example

import asyncio

import ravel


@ravel.interface(ravel.INTERFACE.SERVER, name='com.example.test1')
class Server:
    async def setup(self, loop):
        self.bus = await ravel.session_bus_async(loop)
        self.bus.register('/com/example/test1', False, Server())
        self.bus.request_name(
            'com.example.test1', ravel.DBUS.NAME_FLAG_DO_NOT_QUEUE
        )

    @ravel.method(in_signature='', out_signature='s')
    async def get_string(self):
        return 'hello'

    @ravel.method(in_signature='', out_signature='ss')
    async def get_two_strings(self):
        return 'hello', 'world'

    @ravel.method(in_signature='', out_signature='i')
    async def get_int(self):
        return 1

    @ravel.method(in_signature='', out_signature='ii')
    async def get_two_ints(self):
        return 1, 1

    @ravel.method(in_signature='', out_signature='as')
    async def get_array(self):
        return ['hello', 'world']

    @ravel.method(in_signature='', out_signature='asas')
    async def get_two_arrays(self):
        return ['hello', 'world'], ['goodnight', 'moon']

    @ravel.method(in_signature='', out_signature='a{ss}')
    async def get_dict(self):
        return {'hello': 'world'}

    @ravel.method(in_signature='', out_signature='a{ss}a{ss}')
    async def get_two_dicts(self):
        return {'hello': 'world'}, {'goodnight': 'moon'}

    @ravel.method(in_signature='', out_signature='(ss)')
    async def get_struct(self):
        return 'hello', 'world'

    @ravel.method(in_signature='', out_signature='(ss)(ss)')
    async def get_two_structs(self):
        return ('hello', 'world'), ('goodnight', 'moon')


def main():
    loop = asyncio.get_event_loop()
    server = Server()
    loop.run_until_complete(server.setup(loop))
    try:
        loop.run_forever()
    finally:
        loop.close()


if __name__ == '__main__':
    main()

Single String

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_string
Error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

Task exception was never retrieved
future: <Task finished name='Task-2' coro=<_message_interface_dispatch.<locals>.await_result() done, defined at /home/dhatch/.local/lib/python3.8/site-packages/ravel.py:2006> exception=ValueError("invalid handler result 'hello'")>
Traceback (most recent call last):
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 2016, in await_result
    return_result_common(call_info, result)
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1890, in return_result_common
    raise ValueError("invalid handler result %s" % repr(result))
ValueError: invalid handler result 'hello'

Two Strings

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_two_strings
method return time=1590672580.522903 sender=:1.151623 -> destination=:1.151636 serial=5 reply_serial=2
   string "hello"
   string "world"

Single Int

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_int
Error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

Task exception was never retrieved
future: <Task finished name='Task-4' coro=<_message_interface_dispatch.<locals>.await_result() done, defined at /home/dhatch/.local/lib/python3.8/site-packages/ravel.py:2006> exception=ValueError('invalid handler result 1')>
Traceback (most recent call last):
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 2016, in await_result
    return_result_common(call_info, result)
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1890, in return_result_common
    raise ValueError("invalid handler result %s" % repr(result))
ValueError: invalid handler result 1

Two Ints

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_two_ints
method return time=1590672642.983118 sender=:1.151623 -> destination=:1.151646 serial=6 reply_serial=2
   int32 1
   int32 1

Single Array

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_array
Error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

Task exception was never retrieved
future: <Task finished name='Task-2' coro=<_message_interface_dispatch.<locals>.await_result() done, defined at /home/dhatch/.local/lib/python3.8/site-packages/ravel.py:2006> exception=ValueError("mismatch between signature entries [ArrayType[BasicType(<TYPE.STRING: 115>)]] and number of sequence elements ('hello', 'world')")>
Traceback (most recent call last):
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 2016, in await_result
    return_result_common(call_info, result)
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1892, in return_result_common
    _send_method_return \
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1770, in _send_method_return
    reply.append_objects(sig, *args)
  File "/home/dhatch/.local/lib/python3.8/site-packages/dbussy.py", line 4451, in append_objects
    append_sub(parse_signature(signature), args, self.iter_init_append())
  File "/home/dhatch/.local/lib/python3.8/site-packages/dbussy.py", line 4390, in append_sub
    raise ValueError \
ValueError: mismatch between signature entries [ArrayType[BasicType(<TYPE.STRING: 115>)]] and number of sequence elements ('hello', 'world')

Two Arrays

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_two_arrays
method return time=1590671953.124213 sender=:1.151413 -> destination=:1.151444 serial=5 reply_serial=2
   array [
      string "hello"
      string "world"
   ]
   array [
      string "goodnight"
      string "moon"
   ]

Single Dict

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_dict
Error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

Task exception was never retrieved
future: <Task finished name='Task-4' coro=<_message_interface_dispatch.<locals>.await_result() done, defined at /home/dhatch/.local/lib/python3.8/site-packages/ravel.py:2006> exception=ValueError("invalid handler result {'hello': 'world'}")>
Traceback (most recent call last):
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 2016, in await_result
    return_result_common(call_info, result)
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1890, in return_result_common
    raise ValueError("invalid handler result %s" % repr(result))
ValueError: invalid handler result {'hello': 'world'}

Two Dicts

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_two_dicts
method return time=1590672078.127905 sender=:1.151413 -> destination=:1.151463 serial=6 reply_serial=2
   array [
      dict entry(
         string "hello"
         string "world"
      )
   ]
   array [
      dict entry(
         string "goodnight"
         string "moon"
      )
   ]

Single Struct

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_struct
Error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

Task exception was never retrieved
future: <Task finished name='Task-6' coro=<_message_interface_dispatch.<locals>.await_result() done, defined at /home/dhatch/.local/lib/python3.8/site-packages/ravel.py:2006> exception=ValueError("mismatch between signature entries [StructType((BasicType(<TYPE.STRING: 115>), BasicType(<TYPE.STRING: 115>)))] and number of sequence elements ('hello', 'world')")>
Traceback (most recent call last):
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 2016, in await_result
    return_result_common(call_info, result)
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1892, in return_result_common
    _send_method_return \
  File "/home/dhatch/.local/lib/python3.8/site-packages/ravel.py", line 1770, in _send_method_return
    reply.append_objects(sig, *args)
  File "/home/dhatch/.local/lib/python3.8/site-packages/dbussy.py", line 4451, in append_objects
    append_sub(parse_signature(signature), args, self.iter_init_append())
  File "/home/dhatch/.local/lib/python3.8/site-packages/dbussy.py", line 4390, in append_sub
    raise ValueError \
ValueError: mismatch between signature entries [StructType((BasicType(<TYPE.STRING: 115>), BasicType(<TYPE.STRING: 115>)))] and number of sequence elements ('hello', 'world')

Two Structs

% dbus-send --session --print-reply --dest=com.example.test1 /com/example/test1 com.example.test1.get_two_structs
method return time=1590672175.076293 sender=:1.151413 -> destination=:1.151477 serial=7 reply_serial=2
   struct {
      string "hello"
      string "world"
   }
   struct {
      string "goodnight"
      string "moon"
   }

ravel.Connection.del doesn't reference interfaces as a dict yet

The Connection.__del__ method seems to expect interfaces to be an iterable of objects. However, it appears that interfaces is now a string-indexed dictionary of objects, as other portions of the code indicate.

Currently, I get the following error when starting a Ravel server.[1]

Exception ignored in: <function Connection.__del__ at 0x7ff20c5e2e60>
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/ravel.py", line 258, in __del__
    remove_listeners(self._dispatch, [])
  File "/usr/lib/python3.7/site-packages/ravel.py", line 237, in remove_listeners
    remove_listeners(child, path + [node])
  File "/usr/lib/python3.7/site-packages/ravel.py", line 237, in remove_listeners
    remove_listeners(child, path + [node])
  File "/usr/lib/python3.7/site-packages/ravel.py", line 237, in remove_listeners
    remove_listeners(child, path + [node])
  File "/usr/lib/python3.7/site-packages/ravel.py", line 240, in remove_listeners
    for rulestr in interface.listening :
AttributeError: 'str' object has no attribute 'listening'

This error goes away when I apply the following change that iterates over the interfaces values instead of (implicty) over the string keys of the interfaces dictionary:

--- a/ravel.py
+++ b/ravel.py
@@ -236,7 +236,7 @@ class Connection(dbus.TaskKeeper) :
             for node, child in level.children.items() :
                 remove_listeners(child, path + [node])
             #end for
-            for interface in level.interfaces :
+            for interface in level.interfaces.values() :
                 for rulestr in interface.listening :
                     ignore = dbus.Error.init()
                     self.connection.bus_remove_match(rulestr, ignore)

[1] Yes, the Connection is deleted when I start the Ravel service. I still haven't tracked down why the connection is getting closed almost immediately on startup, but that will be another issue.

Bad bus name is not caught immediately

Issuing the command:

x = ravel.system_bus()["bad_name"]["/"]

does not cause an error although the bus name does not exist. One must try to issue a real operation first, such as x.introspect().

Not sure if the is intended behaviour? This delaying of identify the error makes it more difficult to identify what the problem is as several operations need to be done (naming the bus, naming the path, issuing a request) before an error at any stage is identified.

Should there be a ravel.Connection.listen_signal_async?

Many of the methods have *_async variants, but ravel.Connection.listen_signal does not. It is implemented using dbussy.Connection.bus_add_match which has a bus_add_match_async sibling. That suggests that listen_signal could block. So I'm wondering whether listen_signal is safe to use in an async context. Should there be a listen_signal_async?

Beyond this, I was wondering whether listen_signal could maybe return a context manager that invokes unlisten_signal on __exit__. Would that make sense?

Generated Code?

Just curious.
dbussy's code looks like generated by a tool:

    def iter_init(self) :
        "creates an iterator for extracting the arguments of the Message."
        iter = self.ExtractIter(None)
        if dbus.dbus_message_iter_init(self._dbobj, iter._dbobj) == 0 :
            iter._nulliter = True
        #end if
        return \
             iter
    #end iter_init

Is it?
Or do you just have a very aggressive linter?

Adding to pypi

Hello, I use your library in my project https://github.com/FichteFoll/discordrp-mpris and notice that you don't have it on pypi yet. As you can see in my installation instructions, I can manage without it, but I would love for you to notify me (or everyone else interested) through this issue once you pack up a release and submit it.

I also added it to the AUR as I needed it as a dependency: https://aur.archlinux.org/packages/python-dbussy-git/

Support matching on arg0 for ravel's `listen_signal()`

Hey!

When listening for interface org.freedesktop.DBus.Properties, PropertiesChanged member on some path, it may be useful to further restrict the changes we get by also adding a filter on arg0. This is supported by the low-level library, but the high-level one does not allow further customizations.

Blocking server calls

Hi,
here is the background, just want to see if this is conceptually possible.

I'm in a situation in which I inherited code that is based on callbacks.
It implements it's own event loop hidden somewhere in the underlying libraries.

Is it possible to use dbussy to expose a D-bus connection with blocking code and
expect it as something that will happen quick?

Given that this was a horribly phrased question, here is an example :)

Say that I have a callback new_connection_detected triggered by a new socket connection.
Does dbussy offer a way a blocking way to expose existence of the new connection
as a d-bus interface.
In pseudo code:

def new_connection_detected( connection ):
    debussy.expose("new.connection.1")

The use case is just to have a public exposition of the state through d-bus,
there won't be any interaction happening through d-bus.

Apparently harmless Exception in ravel.py

Hi,
I'm still trying to learn something about dbus and your package.

Using one example, as-is and without changes results in an Exception, but apparently everything still works:

mcon@cinderella:~/projects/dbussy$ cat >t.py
#!/usr/bin/python3
#+
# DBussy example -- make an Introspect query to a specified bus object.
# This version uses the structured high-level Ravel API with asyncio.
# Invoke this script as follows:
#
#    introspect_ravelled «bus» «bus-name» «obj-path»
#
# where «bus» is either “session” or “system”, indicating which bus
# to do the query on, “bus-name” is the name of the service on the
# bus to query, and «obj-path» is the path of the object to query.
# The response should be an XML string in the usual introspection
# format, indicating the protocol for communicating with the
# specified object.
#
# Copyright 2017 by Lawrence D'Oliveiro <[email protected]>. This
# script is licensed CC0
# <https://creativecommons.org/publicdomain/zero/1.0/>; do with it
# what you will.
#-

import sys
import asyncio
import getopt
import dbussy as dbus
from dbussy import \
    DBUS
import ravel

loop = asyncio.get_event_loop()

if len(sys.argv) != 4 :
    raise getopt.GetoptError("usage: %s session|system «bus-name» «obj-path»" % sys.argv[0])
#end if

bus = {"session" : ravel.session_bus, "system" : ravel.system_bus}[sys.argv[1].lower()]()
bus.attach_asyncio(loop)
interface = loop.run_until_complete \
  (
    bus[sys.argv[2]][sys.argv[3]]
        .get_async_interface(DBUS.INTERFACE_INTROSPECTABLE)
  )
reply = loop.run_until_complete(interface.Introspect())
sys.stdout.write(reply[0])

mcon@cinderella:~/projects/dbussy$ python3 t.py system org.bluez /org/bluez
Exception ignored on calling ctypes callback function: <function Connection.add_filter.<locals>.wrap_function at 0x7f85ef860940>
Traceback (most recent call last):
  File "/home/mcon/projects/dbussy/dbussy.py", line 2573, in wrap_function
    result = function(self, message, user_data)
  File "/home/mcon/projects/dbussy/ravel.py", line 2246, in _message_interface_dispatch
    dispatch_signal(bus._dispatch, dbus.split_path(message.pth))
AttributeError: 'Message' object has no attribute 'pth'
<!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<node><interface name="org.freedesktop.DBus.Introspectable"><method name="Introspect"><arg name="xml" type="s" direction="out"/>
</method></interface><interface name="org.bluez.AgentManager1"><method name="RegisterAgent"><arg name="agent" type="o" direction="in"/>
<arg name="capability" type="s" direction="in"/>
</method><method name="UnregisterAgent"><arg name="agent" type="o" direction="in"/>
</method><method name="RequestDefaultAgent"><arg name="agent" type="o" direction="in"/>
</method></interface><interface name="org.bluez.ProfileManager1"><method name="RegisterProfile"><arg name="profile" type="o" direction="in"/>
<arg name="UUID" type="s" direction="in"/>
<arg name="options" type="a{sv}" direction="in"/>
</method><method name="UnregisterProfile"><arg name="profile" type="o" direction="in"/>
</method></interface><node name="hci0"/></node>mcon@cinderella:~/projects/dbussy$

In spite of the error the program still works (and also my own code, a bit more complex works, with exactly the same Exception).

This is completely repeatable for me on two very different systems: an up-to-date Linux-Mint "Ulyssa" and a self-built micro MIPS.
Both systems have the same version of Your code (DBussy-1.3 and ravel-2.0.8, installed via pip not from git)

Trouble sending a message with an argument of type "aay"

Is sending array of arrays of bytes supported?

I get this:
File "/cyc_bsc/scripts/stfs/avahi_wrapper.py", line 69, in start
await self.group.AddService(avahi.IF_UNSPEC, avahi.PROTO_UNSPEC, 0, self.name, self.type, self.domain, self.host, self.port, self.text)
File "/usr/lib/python3.6/site-packages/ravel.py", line 2839, in call_method
_append_args(message, intr_method, args, kwargs)
File "/usr/lib/python3.6/site-packages/ravel.py", line 2776, in _append_args
message.append_objects(call_info.in_signature, *message_args)
File "/usr/lib/python3.6/site-packages/dbussy.py", line 4451, in append_objects
append_sub(parse_signature(signature), args, self.iter_init_append())
File "/usr/lib/python3.6/site-packages/dbussy.py", line 4415, in append_sub
if type_is_fixed_array_elttype(arrelttype.code.value) :
AttributeError: 'ArrayType' object has no attribute 'code'

Where I try to call avahi AddService message with the following signature:

    <method name="AddService">                                                                                          
      <arg name="interface" type="i" direction="in"/>                                                                   
      <arg name="protocol" type="i" direction="in"/>                                                                    
      <arg name="flags" type="u" direction="in"/>                                                                       
      <arg name="name" type="s" direction="in"/>                                                                        
      <arg name="type" type="s" direction="in"/>                                                                        
      <arg name="domain" type="s" direction="in"/>                                                                      
      <arg name="host" type="s" direction="in"/>                                                                        
      <arg name="port" type="q" direction="in"/>                                                                        
      <arg name="txt" type="aay" direction="in"/>                                                                       
    </method>

Seems that the problem is with the last argument.
I tried passing various values to it but nothing works.
Any idea?

New Release 1.x incl. bug fix ae8b2c6

Hello,

This commit seems like a bug fix. https://github.com/ldo/dbussy/tree/ae8b2c622d5c91c1e3bc91cee1e960d518d7a921.

I think it's essential to maintain delivery in mainstream release cleaned and updated.

Do you think you will be able to release soon a new version?

Thank you for your work on it,

B.r,

Clarifying how to listen for signals

I'm having difficulty understanding how to listen for signals using ravel. Consider the following using pydbus:

from gi.repository import GLib
import pydbus

bluez = pydbus.SystemBus().get("org.bluez", "/")
adap  = pydbus.SystemBus().get("org.bluez", "/org/bluez/hci0")

bluez.onInterfacesAdded   = lambda *args: print("InterfacesAdded: ", *args)
bluez.onInterfacesRemoved = lambda *args: print("InterfacesRemoved: ", *args)
adap.onPropertiesChanged  = lambda *args: print("PropertiesChanged: ", *args)

loop = GLib.MainLoop()
loop.run()

This listens for Bluez-related events. I can run bluetoothctl and issue power on, power off, etc. and have the Python script display the resulting signals.

How would I do this using ravel? It is not clear in the documentation what the arguments to listen_signal or the decorator should be. Trying to do something like this doesn't work:

import ravel
import asyncio

@ravel.signal(name = "PropertiesChanged", in_signature = "(s, a{sv}, as)", args_keyword = "args")
def signal_properties_changed(args):
    print(args)

loop = asyncio.get_event_loop()
bus = ravel.system_bus()
bus.attach_asyncio(loop)
bus.listen_signal("/org/bluez/hci0", True, "org.freedesktop.DBus.Properties", "PropertiesChanged", signal_properties_changed)
loop.run_forever()

Exceptions in asynchronous property setting not propagated to the user code

Hi,

When trying to interact with Bluez using dbussy/ravel I found out that errors in asynchronous property setting cannot be properly handled, instead I get error message:

ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<def_proxy_interface.<locals>.def_prop.<locals>.set_prop.<locals>.sendit() done, defined at /usr/local/lib/python3.7/dist-packages/ravel.py:2967> exception=DBusError('org.bluez.Error.Failed -- Not Powered')>
Traceback (most recent call last):
 File "/usr/local/lib/python3.7/dist-packages/ravel.py", line 2977, in sendit
    raise dbus.DBusError(reply.error_name, reply.expect_objects("s")[0])
      dbussy.DBusError: org.bluez.Error.Failed -- Not Powered

In this case I would like to ignore this specific exception, but I cannot, as it is never propagated to my code.
That is because the error is raised from the sendit() coroutine, which is started as an untracked task.

I can see two possible solutions to that:

instead of raising the exception sendit() could set it on the set_prop_pending future (instead of setting its result to None)
alternatively the sendit() tasks could be stored instead of the set_prop_pending futures – that can be awaited the same way

Typo in dbus_connection_remove_filter signature

Hi,

I saw by code review when working on the issue #7 that the dbus_connection_remove_filter argtypes is not set properly.

A patch like this seems needed:

diff --git a/yams/util/dbussy.py b/yams/util/dbussy.py
index ef96067a4cb7..125390920527 100644
--- a/yams/util/dbussy.py
+++ b/yams/util/dbussy.py
@@ -901,7 +901,7 @@ def data_key(data) :
 dbus.dbus_connection_add_filter.restype = DBUS.bool_t
 dbus.dbus_connection_add_filter.argtypes = (ct.c_void_p, ct.c_void_p, ct.c_void_p, ct.c_void_p)
 dbus.dbus_connection_remove_filter.restype = None
-dbus.dbus_connection_add_filter.argtypes = (ct.c_void_p, ct.c_void_p, ct.c_void_p)
+dbus.dbus_connection_remove_filter.argtypes = (ct.c_void_p, ct.c_void_p, ct.c_void_p)
 
 dbus.dbus_connection_allocate_data_slot.restype = DBUS.bool_t
 dbus.dbus_connection_allocate_data_slot.argtypes = (ct.POINTER(ct.c_uint),)

I can create a pull request for this as well if you want.

Adding argument of an Array of Dictionaries leads to AttributeError

When trying to add an array of dictionaries as an argument append_objects() throws an exception:

>>> request = dbussy.Message.new_method_call(..)
>>> request.append_objects("aa{si}", [ { "a": 12, "b": 12 } ])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sag/.local/lib/python3.9/site-packages/dbussy.py", line 4451, in append_objects
    append_sub(parse_signature(signature), args, self.iter_init_append())
  File "/home/sag/.local/lib/python3.9/site-packages/dbussy.py", line 4415, in append_sub
    if type_is_fixed_array_elttype(arrelttype.code.value) :
AttributeError: 'DictType' object has no attribute 'code'

Strange behavior on ravel connection object deletion

Hi, I meet a strange behavior in a kodi plugin using Ravel to handle Bluetooth devices.

The issues that expose the problem and partially resolve it are here:
LibreELEC/LibreELEC.tv#5645
LibreELEC/service.libreelec.settings#245

Basically the code using Ravel is present here:
https://github.com/LibreELEC/service.libreelec.settings/blob/8b1efa30632dd4b5c0493cfab2cc940a927392de/resources/lib/dbus_utils.py#L119-L122

Abbreviated example (it seems to follow your recommendations?):

class LoopThread(threading.Thread):

    def __init__(self, loop):
        super().__init__()
        self.loop = loop
        self.is_stopped = False

    @log.log_function()
    async def wait(self):
        while not self.is_stopped:
            await asyncio.sleep(1)

    @log.log_function()
    def run(self):
        self.loop.run_until_complete(self.wait())
        self.loop.close()

    @log.log_function()
    def stop(self):
        self.is_stopped = True
        self.join()

LOOP = asyncio.get_event_loop()
BUS = ravel.system_bus()
BUS.attach_asyncio(LOOP)
LOOP_THREAD = LoopThread(LOOP)

Calling LOOP_THREAD.stop() is ok but Kodi crashes just after for some reason.

The dbus connection object obtained through Ravel does not seem to be deleted correctly, even if it is explicitly requested with del.
It ends with the following stacktrace after an explicit del BUS:

ERROR <general>: Exception ignored in:
ERROR <general>: <function Connection.__del__ at 0x7f233eb88488>
ERROR <general>:

ERROR <general>: Traceback (most recent call last):
ERROR <general>:   File "/home/pi/.local/lib/python3.7/site-packages/ravel.py", line 287, in __del__
ERROR <general>:
ERROR <general>: remove_listeners(self._client_dispatch, [])
ERROR <general>:
ERROR <general>:   File "/home/pi/.local/lib/python3.7/site-packages/ravel.py", line 268, in remove_listeners
ERROR <general>:
ERROR <general>: for interface in level.interfaces.values() :
ERROR <general>:
ERROR <general>: AttributeError
ERROR <general>: :
ERROR <general>: '_ClientDispatchNode' object has no attribute 'interfaces'

Only a garbage collection solves the problem on the Kodi side.

I am not the developer of this code but I found the way to prevent crash. It is difficult for me to know if it is on the side of dbussy or on the side of the Python code developed on the project.
On the project side it seems to be difficult to know where is the implementation error regarding the dbussy base code.

I am aware that it will also be difficult for you to audit this code base.
However, maybe you can give a hint on what could go wrong and generate the error in Ravel?

Thanks for reading.

StructType inside ArrayType are broken

Hello. I have method with out_signature "a(ii)" and get following exception when it called.
This patch works for me
dbussy.txt

Traceback (most recent call last):
File "_ctypes/callbacks.c", line 234, in 'calling callback function'
File "/home/cctv/.local/lib/python3.6/site-packages/dbussy.py", line 2573, in wrap_function
result = function(self, message, user_data)
File "/home/cctv/.local/lib/python3.6/site-packages/ravel.py", line 2040, in _message_interface_dispatch
return_result_common(call_info, result)
File "/home/cctv/.local/lib/python3.6/site-packages/ravel.py", line 1897, in return_result_common
args = result
File "/home/cctv/.local/lib/python3.6/site-packages/ravel.py", line 1770, in _send_method_return
reply.append_objects(sig, *args)
File "/home/cctv/.local/lib/python3.6/site-packages/dbussy.py", line 4451, in append_objects
append_sub(parse_signature(signature), args, self.iter_init_append())
File "/home/cctv/.local/lib/python3.6/site-packages/dbussy.py", line 4415, in append_sub
if type_is_fixed_array_elttype(arrelttype.code.value) :
AttributeError: 'StructType' object has no attribute 'code'

unlisten_signal error

if rulekey in listeners :
    listeners = listeners[rulekey]
    try :
        listeners.pop(listeners.index(func))
    except ValueError :
        pass
    #end try
    if len(listeners) == 0 :
        ignore = dbus.Error.init()
        self.connection.bus_remove_match \
          (
            _signal_rule(path, fallback, interface, name),
            ignore
          )
        del listeners[rulekey]

That del raises TypeError, because listeners is no longer the dictionary due to the assignment in line 2.

Strange segfault with ctypes

I've run into a strange bug (that may be related to #13) sometimes reproducible by the following test case:

import ravel
import asyncio

async def main():
    async def func():
        bus = ravel.system_bus()
        bus.attach_asyncio()

        manager = await bus['org.bluez']['/'].get_async_interface('org.freedesktop.DBus.ObjectManager')
        asyncio.create_task(manager.GetManagedObjects())

    # Not creating func causes only a "asyncio.base_futures.InvalidStateError: invalid state"
    await func()

    # Any large module works (e.g. aiohttp)
    import numpy

if __name__ == '__main__':
    asyncio.run(main())

It specifically needs Python 3.7.1 for asyncio.create_task and asyncio.run and triggers about 50% of the time on my armv7l server:

Program received signal SIGSEGV, Segmentation fault.
0xb6d45b4c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
(gdb) bt
#0  0xb6d45b4c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.7m.so.1.0
#1  0xb6e43ab8 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.7m.so.1.0
#2  0xb6d6c470 in _PyFunction_FastCallDict () from /usr/lib/libpython3.7m.so.1.0
#3  0xb5edb62c in ?? () from /home/test/venv3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-arm-linux-gnueabihf.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

On other platforms only the following occurs (again with Python 3.7.1):

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 232, in 'calling callback function'
  File "/home/test/venv3/lib/python3.7/site-packages/dbussy.py", line 4725, in _wrap_notify
    function(self, user_data)
  File "/home/test/venv3/lib/python3.7/site-packages/dbussy.py", line 4790, in pending_done
    done.set_result(self.steal_reply())
asyncio.base_futures.InvalidStateError: invalid state

Similar code is part of a much larger application and it doesn't segfault about 30% of the time but with occasional startup errors like:

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 232, in 'calling callback function'
  File "/home/test/venv3/lib/python3.7/site-packages/dbussy.py", line 4725, in _wrap_notify
    function(self, user_data)
NameError: free variable 'function' referenced before assignment in enclosing scope

And:

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 232, in 'calling callback function'
TypeError: _seg_22() takes 0 positional arguments but 2 were given

I've just somewhat figured out how to use the ravel module from the source code and the examples repository, so is this just me misusing it?

type checking in decorator

The @ravel.signal decorator (and others) explicitly checks for the decorated argument to be a types.FunctionType, which means you can't use it to wrap a functools.partial. Can this be relaxed to callable()?

(Specifically I'm trying to do something like:

handler = ravel.signal(...)(functools.partial(self.propchanged, path))
self.bus.listen_propchanged(path, True, None, handler)

since the handlers aren't normally given the object path.)

Cancelled asyncio tasks

I encountered some strange behavior in my setup. I tried to come up with a minimal example, but the behavior varied unexplainably. This is what I've come up with.
The original error I was tracking are the cancelled asyncio tasks. For me the behavior changes for example when changing the number of child levelB's I create. So this looks like some sort of timing/order/race condition issue?
Also, at some point during minifying the DBUS error appeared additionally. Though it probably is a separate issue, as I haven't been able to isolate it, I just sticked with it.
I realize this might very well be some issue with my specific setup/machine/... But for me this is reproducible. Any ideas?
I'm running a current Fedora 29 with D-Bus Message Bus Daemon 1.12.10.

Openning a connection to a separate D-BUS bus

Although the README says we can create a ravel.Connection object, this is discouraged in the class' docstring.

The AT-SPI accessibility system uses a separate bus, which we can get eg. with:

ravel.session_bus()["org.a11y.Bus"]["/org/a11y/bus"]. get_interface("org.a11y.Bus").GetAddress()

This returns a string (well, a 1-element string list, which surprises the dbussy-noob in me) suitable for eg. qdbus --bus .....

Trying to use it with ravel, on Debian/testing (dbussy 1.3):

(gdb) r
Starting program: /usr/bin/python3 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ravel, dbussy
>>> dc = dbussy.Connection("unix:abstract=/tmp/dbus-OKvAHab7BF,guid=16dcea406b353b040999d5dc607c652e")
>>> c = ravel.Connection(dc)

Program received signal SIGSEGV, Segmentation fault.
__GI___pthread_mutex_lock (mutex=0x4600000042) at ../nptl/pthread_mutex_lock.c:67
67      ../nptl/pthread_mutex_lock.c: Aucun fichier ou dossier de ce type.
#0  __GI___pthread_mutex_lock (mutex=0x4600000042) at ../nptl/pthread_mutex_lock.c:67
#1  0x00007ffff6bbc422 in dbus_connection_get_data () from /lib/x86_64-linux-gnu/libdbus-1.so.3
#2  0x00007ffff6bb5269 in ?? () from /lib/x86_64-linux-gnu/libdbus-1.so.3
#3  0x00007ffff6bb5d08 in dbus_bus_get_unique_name () from /lib/x86_64-linux-gnu/libdbus-1.so.3
#4  0x00007ffff77e2d1d in ?? () from /lib/x86_64-linux-gnu/libffi.so.7
#5  0x00007ffff77e2289 in ?? () from /lib/x86_64-linux-gnu/libffi.so.7
#6  0x00007ffff6f21360 in ?? () from /usr/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#7  0x00007ffff6f20e05 in ?? () from /usr/lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
#8  0x000000000051d89b in _PyObject_MakeTpCall ()
#9  0x00000000005175ba in _PyEval_EvalFrameDefault ()
#10 0x0000000000528b63 in _PyFunction_Vectorcall ()
#11 0x0000000000538d46 in ?? ()
#12 0x0000000000526f68 in _PyObject_GenericGetAttrWithDict ()
#13 0x0000000000511ed1 in _PyEval_EvalFrameDefault ()
#14 0x00000000005106ed in ?? ()
#15 0x0000000000528d21 in _PyFunction_Vectorcall ()
#16 0x0000000000572306 in ?? ()
#17 0x000000000051d686 in _PyObject_MakeTpCall ()
#18 0x00000000005175ba in _PyEval_EvalFrameDefault ()
#19 0x00000000005106ed in ?? ()
#20 0x0000000000510497 in _PyEval_EvalCodeWithName ()
#21 0x00000000005f5be3 in PyEval_EvalCode ()
#22 0x0000000000619de7 in ?? ()
#23 0x0000000000615610 in ?? ()
#24 0x0000000000459cb3 in ?? ()
#25 0x0000000000459911 in PyRun_InteractiveLoopFlags ()
#26 0x00000000006194f5 in PyRun_AnyFileExFlags ()
#27 0x000000000044bca9 in ?? ()
#28 0x00000000005ea6e9 in Py_BytesMain ()
#29 0x00007ffff7c4bd0a in __libc_start_main (main=0x5ea6b0, argc=1, argv=0x7fffffffdb48, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdb38)
    at ../csu/libc-start.c:308
#30 0x00000000005ea5ea in _start ()

Given the ctypes in the backtrace it could be linked to #15, but that one is 100% reproducible.

A registered Ravel server can outlive its Bus and Connection

Please consider the following example:

#!/usr/bin/env python3

import asyncio
import dbussy as dbus
from   dbussy import DBUS
import gc
import ravel


@ravel.interface(ravel.INTERFACE.SERVER, name =  "com.example.hello")
class Hello:
    @ravel.method(name          = "hello",
                  in_signature  = "",
                  out_signature = "s")
    def hello(self):
        return ["Hello world!"]


def do_register(loop):
    bus = ravel.session_bus()
    bus.attach_asyncio(loop)
    bus.request_name(bus_name = "com.example.hello",
                     flags    = DBUS.NAME_FLAG_DO_NOT_QUEUE)
    bus.register(path      = "/com/example/hello",
                 fallback  = True,
                 interface = Hello())

loop = asyncio.get_event_loop()
# register our session bus in a method, so that "bus" is not a global that hangs
# around to the end of execution
do_register(loop)

# allow our session bus allocated in do_register() to be garbage collected,
# if nothing else is referencing it
gc.collect()

# start listening for method calls on our Hello server
loop.run_forever()

If we run this DBus service, we will get the following error:

$ python hello_server.py
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 232, in 'calling callback function'
  File "/home/hezekiah/work/didactic/fp/navajo/tmp/dbussy/dbussy.py", line 2555, in wrap_function
    result = function(self, message, user_data)
  File "/home/hezekiah/work/didactic/fp/navajo/tmp/dbussy/ravel.py", line 2117, in _message_interface_dispatch
    assert bus != None, "parent Connection has gone"
AssertionError: parent Connection has gone

This error will occur every time our DBus service receives a message.

As far as I can work out, if the bus object that registers a service object goes out of scope and is garbage collected, the underlying Connection object that is held by the service object as a weak reference is also garbage collected ... but the service object keeps running and continues to receive and try to handle DBus messages (which it fails at because it no longer has a Connection).

The behavior I expected was that either one of the following:

A registered service continues to run and operate, regardless of its registering bus object is destroyed.
A registered service is de-registered and its asyncio Task stopped when its registering bus object is destroyed.