duylam/pi-camera

A web-based video call room for 02 Pi devices installed camera, micro and speaker

pi-camera's Issues

Define protoc buff schema and build Rust app for proxying

Create skeleton folder structure for full solution

Outcome

Below skeleton apps which are runable in local development environment

|- README.md
|- apps
      |- web-viewer-client
      |- pi-camera-client
      |- webrtc-signaling-server

Enhance code skeleton

Checklist

Pi-client

• Logging: namespace, env log level
• TODO note in code
• Dev commands: auto re-format (https://pypi.org/project/autopep8/), create build package

Web-viewer-client

• Refactor structure for nicer and disposable UI
• TODO note in code
• Dev commands: create build package, dockerize

Webrtc-signaling-server

• TODO note in code
• Dev commands: create build package, dockerize

Build web-frontend + pi-camera to send video

Action items in order

• Simulate right behavior for IO.read()
• Refactoring pi client to have clear design for camera, webrtc
• Integrate camera with PoC at https://github.com/aiortc/aiortc/blob/main/examples/webcam/webcam.py
• Use gRPC for signaling between pi and browser
• WebRTC in python (pi):
  • read video spec
  • stream camera to WebRTC
  • compatible video format from camera to MediaPlayer
• signaling server
  • heartbeat
  • forwarding messages
• web-client
  • heartbeat
  • send and receive messages
• pi-client
  • heartbeat
  • send and receive messages
• Verify code running
• Update WebRTC technique on README https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Signaling_and_video_calling

Clean-up for making the project public

The purpose: offer a virtual room connecting 02 physical Pi devices having camera, mic and speaker

Actions:

• remove PI_MEETING name
• update readme.md: introduction, deployment, getting start dev
• convert in queue to Issues
• Make it public
• define roadmap in wiki

PoC: capture Camera and produce playable video file on 3rd video player software

Outcome: put python as cli app for demo and mark as release to save codebase

See WIP code at branch XXXX

Notes for pending next actions

• webrtc
  • aiortc lib, tutorial
• build rasp box for hangout meet (https://gsuite.google.com/products/hangouts-meet-hardware/)
• define blocker part first: stream camera, record and stream voice
  • milestone 1: stream camera from pi to laptop browser
  • milestone 2: capture mic on pi to save to file, then playable on laptop
  • milestone 3: send .wav from file in pi to attached speaker
  • milestone 4: take input on phone to control pi (keyboard + mouse)

Document how to deploy Pi with Camera in office for real usage

• Camera case
• Fan

Bug: pi-client doesn't reconnect grpc after 5s when grpc stream has error (e.g. unavailable to connect)

Stacktrace

2021-08-15 12:42:19,361 pi_box.main_task: [INFO] Started loop
2021-08-15 12:42:20,364 grpc.aio._call: [DEBUG] Exception while consuming the request_iterator: <AioRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses"
	debug_error_string = "{"created":"@1629006140.363326000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3009,"referenced_errors":[{"created":"@1629006140.363322000","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":398,"grpc_status":14}]}"
>
2021-08-15 12:42:20,364 grpc._cython.cygrpc: [DEBUG] Failed to receive any message from Core
2021-08-15 12:42:20,364 pi_box.rtc_task: [ERROR] GRCP stream has error, reconnect again after 5s
Traceback (most rece

Improve PoC at issue #1 to play as middleware for wrapper mp4 container and save to file which is playable in browser

Outcome: updated cli for demo and mark as release to save codebase

Create deploy package with STUN

Outcome

• Real run
  • Stream ok
  • Video is delayed
• Add STUN
• STUN ice in env param
• ICE candidate can't connect 02 peer (pi and web) with AWS server
  • Switch to use coTURN for STUN server
  • Have snippet for using coTURN as STUN client
  • Re-send answer to pi 02, see https://stackoverflow.com/a/38533347