arduino/serial-discovery

An Arduino IDE pluggable-discovery for Serial ports

serial-discovery's Issues

Add macos amd64 architecture

Describe the request

Add support for macos arm64 architecture. Should be a matter of updating the release workflow with the latest modifications from https://github.com/arduino/tooling-project-assets

Describe the current behavior

arduino/arduino-cli#1990

serial-discovery version

1.3.2

Operating system

macOS

Operating system version

Ventura 13.0.1 (22A400)

Additional context

No response

Issue checklist

• I searched for previous requests in the issue tracker
• I verified the feature was still missing when using the latest version
• My request contains all necessary details

Latest serial-discovery release uses version lower than previous

Describe the bug

Prior to arduino/arduino-cli#1537, Arduino CLI installed builtin:[email protected]:
https://github.com/arduino/arduino-cli/pull/1537/files#diff-d3fa6a741630f5c517b99d6c8793c597897132c717d9e7a25cea4320777ab5d4L28

Arduino CLI only installs a tool of the builtin package if it is newer than the latest version already installed:
https://github.com/arduino/arduino-cli/blob/83f3deb14142f6c4eb72992a87dd582c72f97239/arduino/cores/tools.go#L83

The same logic applies to loading the discovery:
https://github.com/arduino/arduino-cli/blob/83f3deb14142f6c4eb72992a87dd582c72f97239/arduino/cores/tools.go#L96

builtin:[email protected] is actually significantly outdated when compared to the latest release builtin:[email protected]:
https://github.com/arduino/serial-discovery/releases
However, since in semver 1.2.2 < 1.3.0-rc1, builtin:[email protected] will never be installed or used so long as the user has builtin:[email protected] installed.

To Reproduce

1. Remove any prior installation of builtin:[email protected]
2. Use Arduino CLI 0.19.3 to run any command that does an init (e.g., arduino-cli board list).
   builtin:[email protected] will be installed if it is not already.
3. Use Arduino CLI 0.20.0 or later to run any command that does an init (e.g., arduino-cli board list).
   builtin:[email protected] will not be installed
4. Use Arduino CLI 0.20.0 or later run a command that runs serial-discovery (e.g., arduino-cli board list)
   builtin:[email protected] will be used.

Demo

$ arduino-cli_0.19.3/arduino-cli board list
Downloading missing tool builtin:[email protected]...
builtin:[email protected] already downloaded
Installing builtin:[email protected]...
builtin:[email protected] installed
Port       Protocol Type    Board Name FQBN Core
/dev/ttyS4 serial   Unknown

$ arduino-cli_nightly/arduino-cli version
arduino-cli  Version: nightly-20211127 Commit: 83f3deb Date: 2021-11-27T01:25:33Z

$ ls ~/.arduino15/packages/builtin/tools/serial-discovery
1.1.0  1.2.1  1.3.0-rc1

$ arduino-cli_nightly/arduino-cli board list  # note that builtin:[email protected] is not installed
Port       Protocol Type    Board Name FQBN Core
/dev/ttyS4 serial   Unknown

$ ls ~/.arduino15/packages/builtin/tools/serial-discovery
1.1.0  1.2.1  1.3.0-rc1

$ rm -rf ~/.arduino15/packages/builtin/tools/serial-discovery/1.3.0-rc1/

$ arduino-cli_nightly/arduino-cli board list  # Now builtin:[email protected] is installed because 1.3.0-rc1 is no longer present
Downloading missing tool builtin:[email protected]...
builtin:[email protected] downloaded
Installing builtin:[email protected]...
builtin:[email protected] installed
Port       Protocol Type    Board Name FQBN Core
/dev/ttyS4 serial   Unknown

$ arduino-cli_0.19.3/arduino-cli board list # Install 1.3.0-rc1 again
Downloading missing tool builtin:[email protected]...
builtin:[email protected] already downloaded
Installing builtin:[email protected]...
builtin:[email protected] installed
Port       Protocol Type    Board Name FQBN Core
/dev/ttyS4 serial   Unknown

$ ls ~/.arduino15/packages/builtin/tools/serial-discovery # Now they are both installed
1.1.0  1.2.1  1.2.2 1.3.0-rc1

$ arduino-cli_nightly/arduino-cli board list --watch & sleep 1; ps aux | egrep "[s]erial-discovery" # Note that 1.3.0-rc1 was used
[1] 6941
Port Type Event Board Name FQBN Core

/dev/ttyS4 Serial Port Connected

per         6949  1.0  0.0 703868  4716 pts/0    Sl   04:05   0:00 /home/per/.arduino15/packages/builtin/tools/serial-discovery/1.3.0-rc1/serial-discovery

Expected behavior

Tool release version numbers to be sequential.

Desktop

• OS: Windows 10, Ubuntu 20.04
• Version: nightly-20211127 Commit: 83f3deb Date: 2021-11-27T01:25:33Z

`serial-discovery` process on Windows consumes lots of CPU

The current version v1.3.0 consumes lots of CPU on Windows, we must find a way to make it more efficient.

Using a FT232R serial to USB adapter breaks serial-discovery

Hi,

Sorry, I am not sure if this an issue with serial-discovery or Raspberry Pi Bullseye AARCH64.

Serial-discovery breaks after plugging in a FT232R serial to USB adapter. Afterwards, serial-discovery will no longer detect Arduino devices on /dev/ttyACM0.

Steps to reproduce:

1. Reboot my Raspberry Pi 400 to make serial-discovery work again
2. Start serial-discovery:

~/.arduino15/packages/builtin/tools/serial-discovery/1.3.0/serial-discovery
HELLO 1 "test"
{
  "eventType": "hello",
  "message": "OK",
  "protocolVersion": 1
}
START_SYNC
{
  "eventType": "start_sync",
  "message": "OK"
}
{
  "eventType": "add",
  "port": {
    "address": "/dev/ttyAMA0",
    "label": "/dev/ttyAMA0",
    "protocol": "serial",
    "protocolLabel": "Serial Port",
    "properties": {}
  }
}
{
  "eventType": "add",
  "port": {
    "address": "/dev/ttyS0",
    "label": "/dev/ttyS0",
    "protocol": "serial",
    "protocolLabel": "Serial Port",
    "properties": {}
  }
}

3. Plugin Arduino UNO R3
   serial-discovery shows new device:

{
  "eventType": "add",
  "port": {
    "address": "/dev/ttyACM0",
    "label": "/dev/ttyACM0",
    "protocol": "serial",
    "protocolLabel": "Serial Port (USB)",
    "properties": {
      "pid": "0x0043",
      "serialNumber": "75033303334351312142",
      "vid": "0x2341"
    }
  }
}

dmesg output:

[  127.366725] usb 1-1.2.1.2: new full-speed USB device number 8 using xhci_hcd
[  127.481544] usb 1-1.2.1.2: New USB device found, idVendor=2341, idProduct=0043, bcdDevice= 0.01
[  127.481563] usb 1-1.2.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=220
[  127.481578] usb 1-1.2.1.2: Manufacturer: Arduino (www.arduino.cc)
[  127.481591] usb 1-1.2.1.2: SerialNumber: 75033303334351312142
[  127.543441] cdc_acm 1-1.2.1.2:1.0: ttyACM0: USB ACM device
[  127.545993] usbcore: registered new interface driver cdc_acm
[  127.546011] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters

4. Disconnect Arduino UNO
   serial-discovery:

{
  "eventType": "remove",
  "port": {
    "address": "/dev/ttyACM0",
    "protocol": "serial"
  }
}

dmesg:

[  229.558923] usb 1-1.2.1.2: USB disconnect, device number 8

5. Reconnect
   So far everything works.
   serial-discovery:

{
  "eventType": "add",
  "port": {
    "address": "/dev/ttyACM0",
    "label": "/dev/ttyACM0",
    "protocol": "serial",
    "protocolLabel": "Serial Port (USB)",
    "properties": {
      "pid": "0x0043",
      "serialNumber": "75033303334351312142",
      "vid": "0x2341"
    }
  }
}

dmesg:

[  291.741001] usb 1-1.2.1.2: new full-speed USB device number 9 using xhci_hcd
[  291.856006] usb 1-1.2.1.2: New USB device found, idVendor=2341, idProduct=0043, bcdDevice= 0.01
[  291.856026] usb 1-1.2.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=220
[  291.856040] usb 1-1.2.1.2: Manufacturer: Arduino (www.arduino.cc)
[  291.856054] usb 1-1.2.1.2: SerialNumber: 75033303334351312142
[  291.861084] cdc_acm 1-1.2.1.2:1.0: ttyACM0: USB ACM device

6. Plugin FT232R
   serial-discovery:

{
  "eventType": "add",
  "port": {
    "address": "/dev/ttyUSB0",
    "label": "/dev/ttyUSB0",
    "protocol": "serial",
    "protocolLabel": "Serial Port (USB)",
    "properties": {
      "pid": "0x6001",
      "serialNumber": "A50285BI",
      "vid": "0x0403"
    }
  }
}

dmesg:

[  386.730155] usb 1-1.2.1.3: new full-speed USB device number 10 using xhci_hcd
[  386.846303] usb 1-1.2.1.3: New USB device found, idVendor=0403, idProduct=6001, bcdDevice= 6.00
[  386.846324] usb 1-1.2.1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  386.846338] usb 1-1.2.1.3: Product: FT232R USB UART
[  386.846350] usb 1-1.2.1.3: Manufacturer: FTDI
[  386.846362] usb 1-1.2.1.3: SerialNumber: A50285BI
[  386.905121] usbcore: registered new interface driver usbserial_generic
[  386.905240] usbserial: USB Serial support registered for generic
[  386.913987] usbcore: registered new interface driver ftdi_sio
[  386.914089] usbserial: USB Serial support registered for FTDI USB Serial Device
[  386.916279] ftdi_sio 1-1.2.1.3:1.0: FTDI USB Serial Device converter detected
[  386.916569] usb 1-1.2.1.3: Detected FT232RL
[  386.929974] usb 1-1.2.1.3: FTDI USB Serial Device converter now attached to ttyUSB0

Now, only the disconnect of ports is detected.
7. Disconnect and reconnect Arduino UNO and disconnect again
serial-discovery:

{
  "eventType": "remove",
  "port": {
    "address": "/dev/ttyACM0",
    "protocol": "serial"
  }
}
{
  "eventType": "remove",
  "port": {
    "address": "/dev/ttyACM0",
    "protocol": "serial"
  }
}

dmesg:

[  485.083088] usb 1-1.2.1.2: USB disconnect, device number 9
[  487.354896] usb 1-1.2.1.2: new full-speed USB device number 11 using xhci_hcd
[  487.469604] usb 1-1.2.1.2: New USB device found, idVendor=2341, idProduct=0043, bcdDevice= 0.01
[  487.469616] usb 1-1.2.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=220
[  487.469621] usb 1-1.2.1.2: Manufacturer: Arduino (www.arduino.cc)
[  487.469626] usb 1-1.2.1.2: SerialNumber: 75033303334351312142
[  487.474346] cdc_acm 1-1.2.1.2:1.0: ttyACM0: USB ACM device
[  490.203490] usb 1-1.2.1.2: USB disconnect, device number 11

The only way to make serial-discovery detect devices again is to reboot.

System details:
Raspberry Pi 400 running Raspberry Pi OS "Bullseye" ARM64:
Linux raspberrypi 5.10.63-v8+ #1488 SMP PREEMPT Thu Nov 18 16:16:16 GMT 2021 aarch64 GNU/Linux

Occasionally, my Arduino Nano 33 BLE is not detected by the gRPC equivalent of `board list watch` after an upload

Describe the problem

From time to time, IDE2 cannot detect the Arduino Nano 33 BLE board after uploading on macOS. I cannot consistently reproduce the defect but what I see is the following error as an INFO from the daemon process:

2022-12-12T08:35:49.545Z daemon INFO INFO[0103] from discovery builtin:serial-discovery received message type: start_sync, message: Error enumerating serial ports: Error while enumerating serial ports: Error extracting port info from device: Property not found: IOCalloutDevice 

The relevant part when the upload is done:

2022-12-12T08:35:48.760Z daemon INFO 258 |  RESP: {
258 |    "out_stream": "d3JpdGUoYWRkcj0weDM0LHNpemU9MHgxMDAwKQp3cml0ZUJ1ZmZlcihzY3JfYWRkcj0weDM0LCBkc3RfYWRkcj0weDE0MDAwLCBzaXplPTB4MTAwMCkKDVs9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT1dIDEwMCUgKDIxLzIxIHBhZ2VzKQ=="
258 |  }

2022-12-12T08:35:48.810Z daemon INFO 258 |  RESP: {
258 |    "out_stream": "CkRvbmUgaW4gMy4zMzIgc2Vjb25kcwpyZXNldCgpCg=="
258 |  }

2022-12-12T08:35:48.862Z daemon INFO 258 |  RESP: {}
258 STREAM CLOSED


2022-12-12T08:35:49.033Z daemon INFO INFO[0102] from discovery builtin:serial-discovery received message type: remove, port: /dev/cu.usbmodem14301 

2022-12-12T08:35:49.033Z daemon INFO 3 |  RESP: {
3 |    "event_type": "remove",
3 |    "port": {
3 |      "port": {
3 |        "address": "/dev/cu.usbmodem14301",
3 |        "protocol": "serial"
3 |      }
3 |    }
3 |  }

2022-12-12T08:35:49.034Z discovery-log INFO {"eventType":"remove","port":{"matchingBoardsList":[],"port":{"address":"/dev/cu.usbmodem14301","label":"","protocol":"serial","protocolLabel":"","propertiesMap":[]}},"error":""}
2022-12-12T08:35:49.036Z root INFO Attached boards and available ports changed:
2022-12-12T08:35:49.036Z root INFO  - Detached board: Arduino Nano 33 BLE [arduino:mbed_nano:nano33ble] from /dev/cu.usbmodem14301 Serial Port (USB)
2022-12-12T08:35:49.036Z root INFO ------------------------------------------

For the record, when this happens no need to reset the board. Starting another board list watch process can detect the board, even if the CLI daemon has lost tracking it in IDE2 after the upload:

Full gRPC log from IDE2: nano33ble_is_not_discovered_after_verify_upload.txt

To reproduce

• Start IDE2,
• Attach the Nano 33 BLE,
• Upload,
• Upload and continue uploading until the discovery fails.

ble_upload_short.mp4

I cannot consistently reproduce the bug.

Expected behavior

• The discovery can reliably detect the board after the upload.
• No from discovery builtin:serial-discovery received message type: start_sync, message: Error enumerating serial ports: Error while enumerating serial ports: Error extracting port info from device: Property not found: IOCalloutDevice error in the logs.

serial-discovery version

1.1.0 or 1.3.2

Operating system

macOS

Operating system version

12.5.1

Additional context

I see this in the daemon logs in IDE2:

INFO[0000] Loading package builtin from: /Users/a.kitta/Library/Arduino15/packages/builtin 
INFO[0000] Checking existence of 'tools' path: /Users/a.kitta/Library/Arduino15/packages/builtin/tools 
INFO[0000] Loading tools from dir: /Users/a.kitta/Library/Arduino15/packages/builtin/tools 
INFO[0000] Loaded tool                                   tool="builtin:[email protected]"
INFO[0000] Loaded tool                                   tool="builtin:[email protected]"
INFO[0000] Loaded tool                                   tool="builtin:[email protected]"
INFO[0000] Loaded tool                                   tool="builtin:[email protected]"
INFO[0000] Loaded tool                                   tool="builtin:[email protected]"

a.kitta@Akoss-MacBook-Pro Desktop % ls -al ~/Library/Arduino15/packages/builtin/tools/serial-discovery
total 0
drwxr-xr-x  4 a.kitta  staff  128 Oct 28 09:19 .
drwxr-xr-x  6 a.kitta  staff  192 Oct  5 14:21 ..
drwxr-xr-x  3 a.kitta  staff   96 Oct 28 09:19 1.1.0
drwxr-xr-x  4 a.kitta  staff  128 Oct  5 14:21 1.3.2

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the latest version
• My report contains all necessary details

Linux discovery finds placeholder /dev/ttyS* devices

When run on Ubuntu 20.04, serial-discovery is finding the /dev/ttyS* placeholder ports.

Looks like the go-serial library is supposed to check for this, but it seems to not be working.
https://github.com/bugst/go-serial/blob/f344e5f764a882b93ba5e38988551cbb2186e948/serial_unix.go#L285

./serial-discovery 
LIST
{
  "eventType": "command_error",
  "message": "First command must be HELLO, but got 'LIST'",
  "error": true
}
HELLO 1 "manual run"
{
  "eventType": "hello",
  "message": "OK",
  "protocolVersion": 1
}
START
{
  "eventType": "start",
  "message": "OK"
}
LIST
{
  "eventType": "list",
  "ports": [
    {
      "address": "/dev/ttyS8",
      "label": "/dev/ttyS8",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS9",
      "label": "/dev/ttyS9",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS0",
      "label": "/dev/ttyS0",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS2",
      "label": "/dev/ttyS2",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS3",
      "label": "/dev/ttyS3",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS7",
      "label": "/dev/ttyS7",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS6",
      "label": "/dev/ttyS6",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyACM0",
      "label": "/dev/ttyACM0",
      "protocol": "serial",
      "protocolLabel": "Serial Port (USB)",
      "properties": {
        "pid": "0x0483",
        "serialNumber": "5393790",
        "vid": "0x16c0"
      }
    },
    {
      "address": "/dev/ttyS1",
      "label": "/dev/ttyS1",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS4",
      "label": "/dev/ttyS4",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    },
    {
      "address": "/dev/ttyS5",
      "label": "/dev/ttyS5",
      "protocol": "serial",
      "protocolLabel": "Serial Port",
      "properties": {}
    }
  ]
}