KeyError: "Couldn't find extension field Devialet.CallMeMaybe.dvltServiceOptions"

[fwestenberg]

KeyError: "Couldn't find extension field Devialet.CallMeMaybe.dvltServiceOptions"

[unusual-thoughts]

Hello,

Hi Floris! This project has been sleeping for a while now, glad to see someone else is interested ;)

It should have all the building blocks needed to do what you want, although it's all quite messy and experimental. The most complete example usage is probably in play_mp3.py, which shows the fairly long series of steps to talk to a devialet device, although it didn't quite work up to actually playing the mp3 for some reason that i forget. Also as you'll see there is a lot of debug information as well as support for packet captures to replay traffic.

First thanks for this impressive piece of work! Personally I'm very interested in creating a Devialet integration for Home Assistant. Especially volume contol is what I'm interested in. Can you please help me getting started with this? First I have the following issue, any idea how to solve this:

Volume control should be the one of the easiest use cases, probably using TooManyFlows.SoundControl.

C:\Repositories\etincelle>python client_test.py
Traceback (most recent call last):
File "client_test.py", line 8, in
from dvlt_client import WhatsUpClient
File "C:\Repositories\etincelle\dvlt_client.py", line 15, in
dvltServiceOptions = dvlt_pool.FindExtensionByName('Devialet.CallMeMaybe.dvltServiceOptions')
KeyError: "Couldn't find extension field Devialet.CallMeMaybe.dvltServiceOptions"

Fixed this error, which is due to my pretty intense usage of internal protobuf python apis (because a lot of custom stuff in the devialet protobuf usage) that didn't mesh well with the cpp backend. Fixed with a hack for now.

Also, do you know if it is possible with this repository to fake a speaker? Because my Devialet Phantom's are on DOS2, I am not able to update my Devialet Dialog unless I have a Phantom (or fake one) with DOS1. Thanks in advance!

It could absolutely be used to fake a speaker. Both speakers and apps implement the same kinds of interfaces, specifically they are both WhatsUp servers and clients
You could register the same services as a regular phantom, with dummy hooks for all the interesting methods, and wait for someone to call you during the discovery and update process, then fill in the blanks as needed. it might help to have a capture of what the real phantom says, and the repo contains tools to analyze and extract all the chatter between app and device so that you can replicate the important parts

[fwestenberg]

Thanks for your quick response! I have been tryin to get a connection, but apparently some things have changed with DOS2. Right now I only use two Phantoms (without dialog). With my Android phone I try to capture the packages from the new Devialet app. But also ports seems to have changed. Both of my Phantoms use different ports for some reason:
Phantom 1: first message on port 37703
Phantom 2: first message on port 35003.

Do you still have your Phantom(s)? What is your configuration?
Thanks again!

[unusual-thoughts]

Yeah ports are random and there are multiple ones for all the services that are exposed. Did you have any luck with my pyhon fix?

[fwestenberg]

I'm trying to get some results now, but nothing satisfying so far. Should I connect to the Phantom directly you think? And what port should I use?

[fwestenberg]

At least the error is gone now ;) But it's just refusing the connection...

[unusual-thoughts]

What is the output ?

[fwestenberg]

The following test:

hostUid = b'etincelle-' + bytes(uuid.uuid4().hex, 'ascii')

discovered = Queue()
dscvr = DevialetDiscovery(discovered serial=hostUid)
dscvr.start()

serial, dvlt_addr = discovered.get(block=True)
print_info(serial, dvlt_addr)

Results in:

�[34mInfo: Searching for Devialet Device...�[0m

And my most positive try so far (port 37703 is always the first port in my trace file):

def callback_test(arg):
    print_data("Callback with", arg)

# Open a WhatsUp connection
wu_client = WhatsUpClient(name="WhatsUp", addr='192.168.1.211', port=37703, start_time=datetime.now())
wu_client.go()
wu_client.service_discovery()

wu_client.keep_receiving(timeout=2)

Results in:

�[34mInfo: Calling method openConnection from service com.devialet.callmemaybe.connection (0)...�[0m
�[32m... with argument:�[0m
�[37m   version: 1�[0m
�[37m   �[0m
�[32mReply:�[0m
�[37m   None�[0m
�[34mInfo: Opened connection to 192.168.1.211 on port 37703�[0m
�[32mList of services:�[0m
�[37m   �[0m
�[31mError: Service name com.devialet.whatsup.registry not in list []�[0m
�[34mInfo: Calling method listServices from service com.devialet.whatsup.registry (0)...�[0m
�[32m... with argument:�[0m
�[37m   �[0m
�[34mInfo: Getting properties for com.devialet.whatsup.registry�[0m
�[31mError: Service name com.devialet.whatsup.registry not in list []�[0m
�[34mInfo: Calling method propertyGet from service com.devialet.whatsup.registry (0)...�[0m
�[32m... with argument:�[0m
�[37m   �[0m
�[33mWarning: Timed out on port 37703�[0m
�[33mWarning: Closing Connection to 192.168.1.211 on port 37703�[0m
Exception in thread WhatsUp:
Traceback (most recent call last):
  File "C:\Users\Floris\AppData\Local\Programs\Python\Python36-32\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Users\Floris\Documents\Repositories\etincelle\dvlt_client.py", line 133, in run
    self.keep_receiving()
  File "C:\Users\Floris\Documents\Repositories\etincelle\dvlt_client.py", line 127, in keep_receiving
    while not self.shutdown_signal and self.receive(timeout):
  File "C:\Users\Floris\Documents\Repositories\etincelle\dvlt_client.py", line 107, in receive
    data = self.sock.recv(2048)

[unusual-thoughts]

I've had a look at the Devialet app binaries, and it looks like the DOS 2 protocol is quite different, for one they are using protobuf 3 now. This repo works with DOS 1 only for now. I haven't updated my phantom, looks like there are only downsides to it (although the spark app does suck)?

[unusual-thoughts]

Can you send me a capture of chatter between your phantom and app ?

[unusual-thoughts]

I'm surprised if they also changed the multicast discovery protocol though, I would still expect to see UDP traffic on port 24242

[fwestenberg]

I sent the capture to your email. When I change to a wifi network without Phantoms, it starts searching with UDP at 224.0.0.251:5353, which seems to be a mDNS discovery(?).

[fwestenberg]

I haven't updated my phantom, looks like there are only downsides to it (although the spark app does suck)?

I think the DOS2 update is fine, Airplay working and Bluetooth (don't know about Spark?). Also nice you don't need a Dialog any more te create a stereo pair. And I use a Sonos connect to get my Phantoms in one app. However, the Phantoms still have a Master volume in my case, so I just need to get this in my Home Assistant app. :-)

[unusual-thoughts]

From your capture, it seems they switched the wire protocol to websockets, and are now using new protobufs for the higher level protocol.
I extracted the new protobufs from the android app, analyzed your capture with https://github.com/mildsunrise/protobuf-inspector, and it looks like the protobufs are Devialet.CallMeMaybe.BaseRequest, with a polymorphic Body

message BaseRequest {
  enum Type {
    InvalidType = 0;
    Request = 1;
    Reply = 2;
    Notification = 3;
  }
  enum SubType {
    InvalidSubType = 0;
    Open = 1;
    Close = 2;
    Call = 3;
    PropertySet = 4;
    PropertyAdd = 5;
    PropertyRemove = 6;
    Exception = 7;
    PropertyReplace = 8;
  }
  .Devialet.CallMeMaybe.BaseRequest.Type type = 1;
  .Devialet.CallMeMaybe.BaseRequest.SubType subType = 2;
  .Devialet.CallMeMaybe.TrackingInfo tracking = 3;
  .Devialet.CallMeMaybe.RequestTarget target = 4;
  .google.protobuf.Any body = 5;
}

which looks a bit like the old

message Request {
  required bytes serverId = 1;
  required fixed32 serviceId = 2;
  required bytes requestId = 3;
  required uint32 type = 4;
  required uint32 subTypeId = 5;
}

message Reply {
  required bytes serverId = 1;
  required fixed32 serviceId = 2;
  required bytes requestId = 3;
  required uint32 type = 4;
  required uint32 subTypeId = 5;
  required sint32 errorCode = 6;
  required bool isMultipart = 7;
}

The chatter looks like:
req

type: Request
subType: Open
tracking {
  id: "\240\205\234\372\337\201O\032\275\202\350b\353\2441\336"
}
target {
  serviceId: 1
}
body: empty

resp

type: Reply
subType: Open
tracking {
  id: "\240\205\234\372\337\201O\032\275\202\350b\353\2441\336"
}
target {
  serviceId: 1
}
body: InterfaceProperties {
    apiVersion: 1
    properties: DaemonRegistryProperties {
        parent: .Devialet.CallMeMaybe.InterfaceProperties empty
        activeInstallation: .Devialet.WhatsUp.InstallationMessage {
            id: bytes (16) "AF A7 EC A5 82 23 52 51 A1 19 52 AC 8A 3E 9F E8"
            name: "Rocket"
        }
        currentAgent: AgentMessage {
            id: AgentIdMessage {
                installationId: bytes (16) (Uuid) "AF A7 EC A5 82 23 52 51 A1 19 52 AC 8A 3E 9F E8"
                hostId: bytes (16) "1E AF 69 D4 FC 85 59 9D 8C 5C FF C1 85 C8 2C 70"
                poolId: bytes (16) "CD 5F 21 AE 8A 06 4D ED 92 0B 69 18 21 88 C3 B5"
            }
            name: "WhatsUp"
            hostName: "SilverPhantom-LXXXXXXXXXXXX"
            serialNumber: "LXXXXXXXXXXXX"
        }
    }
}

[fwestenberg]

What do you think is the best approach? Perhaps create a new repository with a DOS2 API? (finally it should be in a PyPi library).
For a Home Assistant integration I should create an integration with the next services available:

• turn_on
• turn_off
• toggle
• volume_up
• volume_down
• volume_set
• volume_mute (volume 0 and restore should work)
• media_play_pause (toggle function)
• media_play
• media_pause
• media_stop
• media_next_track
• media_previous_track
• clear_playlist
• shuffle_set
• play_media
• select_source
• select_sound_mode

Bold are most interested to me, I would prefer to start with this. Some functions don't need to be available for Devialet.

[fwestenberg]

Hi Henri, Is there any chance you can help me to create a working volume control for the Phantoms? I would be really happy if so! Please let me know. Kind regards, Floris.

[fwestenberg]

Hi Henri, I hope you are doing well. At this time, I have some time to spend on the Phantom volume control API. Perhaps you are willing to help me a little further with this? I'm stuck on how the chatter should happen, so just for the volume control. Thanks, I hope to hear from you soon! Kind regards, Floris. Op wo 17 jun. 2020 23:21 schreef Floris Westenberg <[email protected]>:

…

Hi Henri, Is there any chance you can help me to create a working volume control for the Phantoms? I would be really happy if so! Please let me know. Kind regards, Floris. Op zo 29 mrt. 2020 18:05 schreef Henri Chain ***@***.***>: > From your capture, it seems they switched the wire protocol to > websockets, and are now using new protobufs for the higher level protocol. > I extracted the new protobufs from the android app, analyzed your capture > with https://github.com/mildsunrise/protobuf-inspector, and it looks > like the protobufs are Devialet.CallMeMaybe.BaseRequest, with a > polymorphic Body > > message BaseRequest { > enum Type { > InvalidType = 0; > Request = 1; > Reply = 2; > Notification = 3; > } > enum SubType { > InvalidSubType = 0; > Open = 1; > Close = 2; > Call = 3; > PropertySet = 4; > PropertyAdd = 5; > PropertyRemove = 6; > Exception = 7; > PropertyReplace = 8; > } > .Devialet.CallMeMaybe.BaseRequest.Type type = 1; > .Devialet.CallMeMaybe.BaseRequest.SubType subType = 2; > .Devialet.CallMeMaybe.TrackingInfo tracking = 3; > .Devialet.CallMeMaybe.RequestTarget target = 4; > .google.protobuf.Any body = 5; > } > > which looks a bit like the old > > message Request { > required bytes serverId = 1; > required fixed32 serviceId = 2; > required bytes requestId = 3; > required uint32 type = 4; > required uint32 subTypeId = 5; > } > > message Reply { > required bytes serverId = 1; > required fixed32 serviceId = 2; > required bytes requestId = 3; > required uint32 type = 4; > required uint32 subTypeId = 5; > required sint32 errorCode = 6; > required bool isMultipart = 7; > } > > The chatter looks like: > *req* > > type: Request > subType: Open > tracking { > id: "\240\205\234\372\337\201O\032\275\202\350b\353\2441\336" > } > target { > serviceId: 1 > } > body: empty > > *resp* > > type: Reply > subType: Open > tracking { > id: "\240\205\234\372\337\201O\032\275\202\350b\353\2441\336" > } > target { > serviceId: 1 > } > body: InterfaceProperties { > apiVersion: 1 > properties: DaemonRegistryProperties { > parent: .Devialet.CallMeMaybe.InterfaceProperties empty > activeInstallation: .Devialet.WhatsUp.InstallationMessage { > id: bytes (16) "AF A7 EC A5 82 23 52 51 A1 19 52 AC 8A 3E 9F E8" > name: "Rocket" > } > currentAgent: AgentMessage { > id: AgentIdMessage { > installationId: bytes (16) (Uuid) "AF A7 EC A5 82 23 52 51 A1 19 52 AC 8A 3E 9F E8" > hostId: bytes (16) "1E AF 69 D4 FC 85 59 9D 8C 5C FF C1 85 C8 2C 70" > poolId: bytes (16) "CD 5F 21 AE 8A 06 4D ED 92 0B 69 18 21 88 C3 B5" > } > name: "WhatsUp" > hostName: "SilverPhantom-LXXXXXXXXXXXX" > serialNumber: "LXXXXXXXXXXXX" > } > } > } > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#1 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ALNVVJ6L6GULTRHYRMD6KILRJ5WTXANCNFSM4LU6DFGA> > . >