Comments (11)
Hey @daya, definitely it's possible. Checkout the android or ios example https://github.com/elixir-desktop/ios-example-app in those you also have platform code in the Bridge.swift
/ Bridge.kt
files there you can access all hardware functions and pass them to your Elixir app.
from desktop.
-
For the camera specifically, but maybe for other services as well it might make sense to use the HTML/Javascript API of these from within the Browser instead of using the path from Bridge to Elixir. Something to try (https://stackoverflow.com/questions/6336641/camera-access-through-browser)
-
On the wx side I don't really see any good mappings. So I would go with something new. Probably create a new module file called "BridgeServices.ex
or such and add the new functions there, so that they can work for both iOS and Android, the functions would of course just forward to the Bridge communication like all other wx* functions at the moment there. On the other side of this in the
Bridge.ktand
Bridge.swift` then catch those calls and fetch the corresponding data.
I would for now stay away from Ports and NIFs as those are a pain to compile for all the mobile platforms, but rather try to solve with existing Bridge. For live Video streaming the current bridge might now work though as it assumes JSON encoding (which makes binary data much larger) and it's not suited for large payloads like a video stream. For a video stream I would open a new TCP or UDP port and use the JSON Bridge communication just to agree on the ports used... But I'm getting ahead of myself.
from desktop.
- for other services as well it might make sense to use the HTML/Javascript API of these from within the Browser instead
That sounds reasonable but probably won't support older versions of mobile OS i.e. before Apple/Google started supporting these HTML APIs. Therefore going React Native route may be a better option but then one will need
- Creating a release that packages
elixir-desktop
along withReact Native
code - And some kind of bridge between Elixir and React Native code
JSON encoding (which makes binary data much larger) and it's not suited for large payloads like a video stream
The new bridge doesn't have to use JSON it could simply use other more efficient protocols e.g. RTP, RTMP, WebRTC or even RTSP, but since the distance between the client & server will be just few silicons then JSON may not be a problem after all.
from desktop.
since the distance between the client & server will be just few silicons then JSON may not be a problem after all.
btw: It doesn't matter on distance in this case, it can be issue with encoding/decoding time on same device. I assume that it uses http protocol under the hood, so there is multiple encodings/decodings for example:
some server type -> json -> http "string" -> json -> some client type
Using some other protocol can save you some en/decodings and it can save you time and batter when we are talking about mobile devices. But I think this problem doesn't have to be solved now :)
btw good job with this project, thanks :)
from desktop.
@dominicletz Looking at the Bridge code it seems its based on and limited by wxWidgets capabilities, right? If yes, then its fair to conclude that mobile device specific functions like accessig camera or finger print sensor or location services won't be mapped by Bridge any time soon. Is that correct?
Is there any alternative approach or alternative Bridge implementation in the works ?
from desktop.
In our primary app we don't need these at the moment, so yes it's unlikely I will add those soon. But I'm happy to take any and all pull requests in this direction.
from desktop.
@dominicletz I will be more than happy to create a PR, but I will need your guidance to familiarize with few concepts
Basically I want to know how should one think about this undertaking to support Camera, Location Services, Bio sensors.
-
Is there a better or alternative to the Bridge approach ? i.e. Ports and NIFs?
1.1. Will the Bridge approach work for things like Camera or is it only good for URL navigation, launching browser, sending notifications etc ? -
To start with should I identify the equivalent wxWidget function for Camera etc ?
2.1. If yes, could you please point me to the some code samples showing this pattern ? as I am having hard time figuring this out end to end. FYI I have looked atBridge.swift
andBridge.kt
and I can see themethod
name pattern inhandle
function but how is Elixir side of code using these methods ?
2.2. If no, then does it mean wxWidget compatibility/mapping is not required ? If yes, then we still need to map to some common set on Elixir side to support both Android and iOS, so what would that common set of functions be?
from desktop.
Yes, encoding exactly. For Video content especially if it's a ton of it we should definitely skip any json encoding, and instead just push the raw bytes to save CPU cycles and battery on the phones.
The comment about react-native I don't understand but then I also don't know anything about react-native. I personally definitely stay away from as much javascript development as possible, so no plans from my side to include an additional js framework. That said if there is a pull request that allows us more access to hardware from the Elixir apps and happens to use react-native I wouldn't block it either.
Cheers!
from desktop.
Well, streaming video is a little bit complicated and different and I think you should check some solution for Elixir and Phoenix itself outside of this elixir-desktop "solution". For example there some older posts about it
https://littlelines.com/blog/2020/07/06/building-a-video-chat-app-in-phoenix-liveview
https://dockyard.com/blog/2020/09/25/live-streaming-with-liveview-and-mux-in-under-70-lines-of-code
from desktop.
I think you should check some solution for Elixir and Phoenix itself outside of this elixir-desktop "solution".
But how will that help with a mobile application that allows user to upload videos ? If a web app is rendered in a WebView then one is left to the mercy of OS to support HTML5 tags or hope JS hackery works.
It seems the only reliable way is to use device OS native API to access camera.
from desktop.
That's true but still you have two battle fronts:
- OS native API for camera access
- accept data in Elixir from native API
from desktop.
Related Issues (20)
- M1 / Apple Silicon / ARM compatibility HOT 2
- blurry text on high DPI monitors (Windows) HOT 4
- Project Roadmap? HOT 1
- Incorrect types in menu.ex fire up dialyzer HOT 2
- iOS app HOT 1
- Using database with it? HOT 4
- Unknown function error for wxWebView.new when starting application on linux HOT 8
- Recurring error "key :todos not found in: %Desktop.Menu" HOT 1
- GLib-CRITICAL Failed to set scheduler settings: Operation not permitted on Ubuntu HOT 1
- LiveView 0.18 not supported HOT 6
- Is there a way to have 2 icons when using elixir-desktop? HOT 1
- iOS Example can't reconnect at real device HOT 1
- Notifications not showing on OSX HOT 2
- Changing endpoint.ex to "use Desktop.Endpoint" breaks Desktop app HOT 2
- Desktop App does not close and always reopens HOT 8
- Releasing & Distributing
- Packaging - any guides? HOT 2
- How to start contributing HOT 1
- Menu / MenuBar not being displayed properly HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from desktop.