mavlink/mavlink-camera-manager

MAVLink Camera Manager Service

MAVLink

MAVLink -- Micro Air Vehicle Message Marshalling Library.

MAVLink is a very lightweight, header-only message library for communication between drones and/or ground control stations. It consists primarily of message-set specifications for different systems ("dialects") defined in XML files, and Python tools that convert these into appropriate source code for supported languages. There are additional Python scripts providing examples and utilities for working with MAVLink data.

Tip MAVLink is very well suited for applications with very limited communication bandwidth. Its reference implementation in C is highly optimized for resource-constrained systems with limited RAM and flash memory. It is field-proven and deployed in many products where it serves as interoperability interface between components of different manufacturers.

Quick start

Generate C headers

To install the minimal MAVLink environment on Ubuntu LTS 20.04 or 22.04, enter the following on a terminal:

# Dependencies
sudo apt install python3-pip

# Clone mavlink into the directory of your choice
git clone https://github.com/mavlink/mavlink.git --recursive
cd mavlink

python3 -m pip install -r pymavlink/requirements.txt

You can then build the MAVLink2 C-library for message_definitions/v1.0/common.xml from the /mavlink directory as shown:

python3 -m pymavlink.tools.mavgen --lang=C --wire-protocol=2.0 --output=generated/include/mavlink/v2.0 message_definitions/v1.0/common.xml

Use from cmake

To include the headers in cmake, install them locally, e.g. into the directory install:

cmake -Bbuild -H. -DCMAKE_INSTALL_PREFIX=install -DMAVLINK_DIALECT=common -DMAVLINK_VERSION=2.0
cmake --build build --target install

Then use find_package to get the dependency in CMakeLists.txt:

find_package(MAVLink REQUIRED)

add_executable(my_program my_program.c)

target_link_libraries(my_program PRIVATE MAVLink::mavlink)

And pass the local install directory to cmake (adapt to your directory structure):

cd ../my_program
cmake -Bbuild -H. -DCMAKE_PREFIX_PATH=../mavlink/install

For a full example, check examples/c.

Note: even though we use target_link_libraries in cmake, it doesn't actually "link" to MAVLink as it's just a header-only library.

Other instructions

Instructions for using the C libraries are then covered in Using C MAVLink Libraries (mavgen).

Note: Installing the MAVLink Toolchain explains how to install MAVLink on other Ubuntu platforms and Windows, while Generating MAVLink Libraries explains how to build MAVLink for the other programming languages supported by the project. The sub-topics of Using MAVLink Libraries explain how to use the generated libraries.

Key Links

• Documentation/Website (mavlink.io/en/)
• Discussion/Support
• Contributing
• License

mavlink-camera-manager's Issues

enable cors

QGC integration is failing

Changes on QGC control does not affect v4l controls

Add wrapper to print every REST API call for debug

It may be something accessible via the logger interface from our library

MAVlink information needs to be updated to support both RTSP and RTPUDP depending of the video stream

https://github.com/patrickelectric/mavlink-camera-manager/blob/ed527463feef50617012ec88106a04035542be86/src/mavlink/mavlink_camera.rs#L537

Blue Robotics' low light hd usb camera keeps restarting

The stream keeps on restarting, and we only get a few frames at a time.
It seems to be caused by patrickelectric@b31c6fb.
t3.0.1 works fine.

Pass the correct information of the streams to mavlink

https://github.com/bluerobotics/mavlink-camera-manager/blob/510a6c4eddd55fb68d2b252222b36011715e9f63/src/mavlink/mavlink_camera.rs#L542-L546

Add/define license

Quick link: https://github.com/bluerobotics/mavlink-camera-manager/community/license/new?branch=master&template=mit

Advertising wrong IP for XML files causes QGroundcontrol to be unable to know camera controls.

note: 192.168.42.1 is the hotspot IP since 1.1.0-beta.5 (Release Candidate)

make fake source not be provided as an available camera or source stream

Fake source should be created by a post and a stream should be available for it, but it should not appears in v4l

Panic when loading bluerov udp

Because Stepwise is not supported yet, the sizes vector can be empty:

WARN  13:55:38.843 src/video/video_source_local.rs:163: Stepwise framesize not suppported for camera: /dev/video0, for configuration: FrameSize {
    index: 0,
    fourcc: FourCC {
        repr: [
            82,
            88,
            50,
            52,
        ],
    },
    typ: 3,
    size: Stepwise(
        Stepwise {
            min_width: 32,
            max_width: 2592,
            step_width: 2,
            min_height: 32,
            max_height: 1944,
            step_height: 2,
        },
    ),
}
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/custom/bluerov.rs:30:37
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Steps to reproduce:

1. Enable the legacy camera mode via red-pill->raspi-config, reboot
2. Boom, panic.

swagger documentation is missing version and title field

{
  "swagger": "2.0",
  "definitions": {},
  "paths": {},
  "info": {
    "version": "",
    "title": ""
  }
}

rtsp: add it back

reporting frame size twice

by @Williangalvani

No errors when creating a stream with wrong Size/FPS

Related to #78.

When creating a stream using the wrongly associated Size/FPS for YUYV (eg. 1920x1080@30FPS), no errors are returned, and neither new stream populates the Stream list, but when trying to create another stream, it returns two possible errors:

• Stream name already in use
• Device already in use

Thus, indicating a bug on the back-end side.

rest api endpoint should be created via command line argument

Add webrtc server

https://rustrepo.com/repo/centricular-webrtcsink-rust-network-programming

Improve access to webrtc webpage

We can also merge both pages in a single one

Send stats of the webrtc

https://youtu.be/eJpxqVr_tzQ
https://github.com/centricular/webrtcsink/tree/main/plugins/examples

Move UsbBus from struct to string

A struct is problematic since the type of the information can be a tree of usb bus of usb bus.
we are going to change to a string to be more generic.

Front-end shows wrong Size/FPS

For this specific camera, max. YUYV is 800x600@15FPS, but because MJPG accepts 1920x1080@30FPS, the Size and FPS lists get messed, wrongly associating YUYV with 1920x1080@30FPS. See the video below:

bug_size.mp4

Nothing happens when trying to configure stream without endpoint

Configuring a stream without an endpoint on the HTML interface results in nothing. No stream created and no warns to the user.

settings with multiple streams configured aren't loaded correctly

See attached log.
cameramanager.txt

In master.rs:33, both listed streams start showing as having the same video_source. the code seems to think that the camera moved.

Expose "inactive" flag for controls

root@companion:/# v4l2-ctl --list-ctrls -d2
                     brightness 0x00980900 (int)    : min=-64 max=64 step=1 default=0 value=0
                       contrast 0x00980901 (int)    : min=0 max=64 step=1 default=32 value=32
                     saturation 0x00980902 (int)    : min=0 max=128 step=1 default=56 value=56
                            hue 0x00980903 (int)    : min=-40 max=40 step=1 default=0 value=0
 white_balance_temperature_auto 0x0098090c (bool)   : default=1 value=1
                          gamma 0x00980910 (int)    : min=72 max=500 step=1 default=100 value=100
                           gain 0x00980913 (int)    : min=0 max=100 step=1 default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=2 default=1 value=1
      white_balance_temperature 0x0098091a (int)    : min=2800 max=6500 step=1 default=4600 value=4600 flags=inactive
                      sharpness 0x0098091b (int)    : min=0 max=6 step=1 default=3 value=3
         backlight_compensation 0x0098091c (int)    : min=0 max=2 step=1 default=1 value=1
                  exposure_auto 0x009a0901 (menu)   : min=0 max=3 default=3 value=3
              exposure_absolute 0x009a0902 (int)    : min=1 max=5000 step=1 default=156 value=156 flags=inactive
         exposure_auto_priority 0x009a0903 (bool)   : default=0 value=0

Exposing the "inactive" flag would allow frontends to deactivate controllers when they are disabled (for example exposure control when in automatic exposure mode).

Controls don't work for LegacyRpiCam

v4l2 controls don't take effect on rpicamsrc streams.

We need to enforce the impl of drop by video streams

Frontend fails to show available streams with old settings

warning: unused manifest key: package.path
   Compiling mavlink-camera-manager v0.2.4 (/home/patrick/git/patrick/ardupilot-camera-manager)
    Finished dev [unoptimized + debuginfo] target(s) in 19.92s
     Running `target/debug/mavlink-camera-manager`
INFO  15:51:24.017 src/logger/manager.rs:43: mavlink-camera-manager, version: 0.2.4-518535c, build date: 2022-02-07
INFO  15:51:24.017 src/logger/manager.rs:50: Starting at 2022-02-07T15:51:24
INFO  15:51:24.018 /home/patrick/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-server-1.0.4/src/builder.rs:263: Starting 16 workers
INFO  15:51:24.020 /home/patrick/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-server-1.0.4/src/builder.rs:277: Starting "actix-web-service-0.0.0.0:6020" service on 0.0.0.0:6020
[src/stream/stream_backend.rs:319] &pipeline = "videotestsrc pattern=ball ! video/x-raw,width=720,height=480,framerate=60/1 ! videoconvert ! x264enc bitrate=5000 ! video/x-h264, profile=baseline ! h264parse ! queue ! rtph264pay name=pay0 config-interval=10 pt=96"
INFO  15:51:24.020 src/stream/stream_backend.rs:320: Created pipeline: videotestsrc pattern=ball ! video/x-raw,width=720,height=480,framerate=60/1 ! videoconvert ! x264enc bitrate=5000 ! video/x-h264, profile=baseline ! h264parse ! queue ! rtph264pay name=pay0 config-interval=10 pt=96
[src/stream/stream_backend.rs:324] &stream = VideoStreamRtsp {
    pipeline: Pipeline {
        description: "videotestsrc pattern=ball ! video/x-raw,width=720,height=480,framerate=60/1 ! videoconvert ! x264enc bitrate=5000 ! video/x-h264, profile=baseline ! h264parse ! queue ! rtph264pay name=pay0 config-interval=10 pt=96",
    },
    endpoint_path: "/test0",
}

/streams

[
   {
      "running":true,
      "video_and_stream":{
         "name":"ball - Fake source",
         "stream_information":{
            "endpoints":[
               "rtsp://0.0.0.0:8554/test0"
            ],
            "configuration":{
               "encode":"H264",
               "height":480,
               "width":720,
               "frame_interval":{
                  "numerator":1,
                  "denominator":60
               }
            },
            "extended_configuration":null
         },
         "video_source":{
            "Gst":{
               "name":"Fake source",
               "source":{
                  "Fake":"ball"
               }
            }
         }
      }
   }
]

url of xml in the mavlink message is missing port

Add command line option to filter some mavlink messages

The verbose mode with mavlink messages is pretty crazy, it would help if we could add an option with the verbose that we could filter the selected mavlink messages with a regex for names.

Implement MAV_CMD_RESET_CAMERA_SETTINGS

The command MAV_CMD_RESET_CAMERA_SETTINGS(529) is used by QGC:

Add option to set camera as thermal

Move supported schemes to ENUM

The schemes that we are using, such as: "udp" | "udp265"| "rtsp" | "mpegts" | "tcp", should be enums to deal with it in a safer way

About the duplicated device

What seems to be 'duplicated devices' on our frontend is actually somewhat an extended feature from Linux Kernel.

Because the user can't use (I'm assuming this based on my tests) both videos from an unique bus, and the commands from one affects the others, I think we should deal with it, and use it in a more concise way.

My initial thought on that is that maybe we could merge all the /dev/video* that shares the same bus, but probably we can't simply merge the controls/capabilities and expect it to works using the devices interchangeably - lets say video0 has JPEG encode, and to the video2 has H264, but if we merge both and keep only the video0 reference, can we use video0 as H264? Probably not, then we would need to add an abstraction layer just to map each capability to its original device (in this example, map JPEG to video0 and H264 to video2).

Maybe we could change our model to something that encapsulates each device inside each bus:

{
'': {
    'bus': 'usb-0000:09:00.3-2.4',
    'devices': [
      {
        'name': 'Video Capture 6',
        'device_path': '/dev/video2',
        ...
      },
      {
        'name': 'Video Capture 5',
        'device_path': '/dev/video0',
        ...
      }
    ]
    'controls': ...,
    'formats': ...,
    ...
  }
}

Do we want to deal with it, or we'll let it to the user? Any ideas?

move to travis since github actions does not have a native arm machine

Wrong autopilot type on heartbeat messages.

As stated in https://mavlink.io/en/messages/common.html#HEARTBEAT, we should use MAV_AUTOPILOT_INVALID, not MAV_AUTOPILOT_GENERIC.

Add support for manual video source/stream

Control via mavlink messages using QGC master is broken

RROR 12:16:43.188 src/mavlink/mavlink_camera.rs:341: Failed to parse control id: ParseIntError {
    kind: InvalidDigit,
}
DEBUG 12:16:43.438 src/mavlink/mavlink_camera.rs:199: sending heartbeat
ERROR 12:16:43.742 src/mavlink/mavlink_camera.rs:341: Failed to parse control id: ParseIntError {
    kind: InvalidDigit,
}
ERROR 12:16:44.340 src/mavlink/mavlink_camera.rs:341: Failed to parse control id: ParseIntError {
    kind: InvalidDigit,
}
DEBUG 12:16:44.439 src/mavlink/mavlink_camera.rs:199: sending heartbeat
DEBUG 12:16:45.439 src/mavlink/mavlink_camera.rs:199: sending heartbeat
ERROR 12:16:46.322 src/mavlink/mavlink_camera.rs:341: Failed to parse control id: ParseIntError {
    kind: InvalidDigit,
}
DEBUG 12:16:46.439 src/mavlink/mavlink_camera.rs:199: sending heartbeat
ERROR 12:16:46.822 src/mavlink/mavlink_camera.rs:341: Failed to parse control id: ParseIntError {
    kind: InvalidDigit,
}
ERROR 12:16:47.322 src/mavlink/mavlink_camera.rs:341: Failed to parse control id: ParseIntError {
    kind: InvalidDigit,
}

Create sdp files for stream (Open stream on VLC)

This helps vlc and others programs to run streams without much user configuration

Suggestion: Control components extend basic control type

Today, the structure (typescript version) of the controls are something like this:

export interface V4LMenuOption {
  name: string
  value: number
}

export interface V4LMenu {
  default: number
  value: number
  options: V4LMenuOption
}

export interface V4LBool {
  default: number
  value: number
}

export interface V4LSlider {
  default: number
  max: number
  min: number
  step: number
  value: number
}

export interface V4LConfiguration {
  Slider?: V4LSlider
  Menu?: V4LMenu
  Bool?: V4LBool
}

export interface V4LControl {
  configuration: V4LConfiguration
  cpp_type: string
  id: number
  name: string
}

The problem with this structure is that we have to check if the component we want (slider, menu or bool) actually exist before consuming it.

if (control.configuration.Slider !== undefined) {
  # use slider data
}

A suggestion would be to remove the V4LConfiguration and make Slider, Menu and Bool extend from V4LControl.

Add support for raspberry camera

original comment: bluerobotics/BlueOS#991

Move code to use format args implicit

https://rust-lang.github.io/rfcs/2795-format-args-implicit-identifiers.html
It's available on current rust stable

Add option to enable/disable mavlink

bluerobotics/BlueOS#1010

Save logs to appropriate place

Logs should be saved under /var/logs/blueos/.

Suggestion: Use default endpoint path when no endpoint is passed

It would be handy to the user being able to create a stream without typing an endpoint path.

add endpoint to reset settings

Reset settings, use default option and restart service

v4l endpoint should no handle reset ou set control

should be a generic endpoint for any kind of camera

Add check and warning message for missing webrtcsink

Also disable webrtcsink pipeline creation if the element is not avaiable

Failing to load settings between versions

for more information: bluerobotics/BlueOS#1030

Allow user to create pipeline video sources

Add gst as feature

gst should not be standard

Update paperclip to the latest release

It already contains the necessary changes