Q: Can I place HTML elements (buttons and so on) on top of active </

Not sure if it'll work with a video layer <p dir="auto"

Drawing HTML on top of a UIView is possible,about cordova-rtc/cordova-plugin-iosrtc

Comments (60)

alexp25 commented on May 13, 2024 1

Is it possible to hide the video tag from view? This way I imagine it would be possible to show the video in canvas while the original video is playing in the background, hidden from the viewport

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

The aim of the cordova-plugin-iosrtc is to provide the WebRTC API without requiring custom stuff. If the video element is placed behind the Cordova HTML UIView and the background of the HTML view is not transparent, then the video would not be shown. How would that work if a <body> background is desired (for example a bitmap texture or a gradient)?

Said that, I may add an option to set this behavior per <video> element. Any suggestion? Perhaps a custom CSS field? a custom data-xxxx="true" attribute?

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Not sure if it'll work with a video layer

Yes, sure.

from cordova-plugin-iosrtc.

telemakhos commented on May 13, 2024

How would that work if a background is desired (for example a bitmap texture or a gradient)?

Users can still set a desired background. Here's how the gmaps plugin solves the issue:

Map.setBackgroundColor()

Yes, sure.

That's great news then. Might be worth exploring more in depth this option...

Any suggestion?

Not really, you know better than anyone what solution would fit better in your project and I just know the basic internals of the gmaps plugin. As a user I think that gmaps makes a great job providing a dummy transparent layer where you can draw freely any HTML element. That'd be undoubtedly the best solution since the users could draw a proper UI on top of the vid elements...

In any case I'm going to continue researching about this issue. I'll get back to you if I find anything worth looking at or if I have any interesting idea...

from cordova-plugin-iosrtc.

murillo128 commented on May 13, 2024

How about?

z-index: -1;

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Using "z-index: -1" is what I consider a non appropriate usage of the CSS property. This is, setting "z-index: -1" may have very different results when in a web browser with native WebRTC support and when in an iOS Cordova device.

I would prefer having a custom data-xxxx attribute that would be ignored in the desktop browser or Android browser:

<video data-iosrtc-rear="true" autoplay></video>

Thoughts?

from cordova-plugin-iosrtc.

murillo128 commented on May 13, 2024

We are already using the z-index in a non-appropriate way to place the videos relative to other videos, so " may have very different results when in a web browser with native WebRTC" is already happening.

from cordova-plugin-iosrtc.

murillo128 commented on May 13, 2024

Also, that way we ensure that the app has to put a transparent background in order the video to be seen.. ;)

from cordova-plugin-iosrtc.

telemakhos commented on May 13, 2024

The issue targeting CSS to only iOS (or web or Android) has several solutions. Ionic uses classes like…

.platform-browser video_div { z-index: x; }
.platform-android video_div { z-index: y; }
.platform-ios video_div { z-index: z; }

and also allows you to use dynamic targeted CSS using platform and ng-class (this is for angular but can be done with vanilla js).

The main issue here is abstracting everything and letting the users easily draw custom controls. I think this could be a very valuable differentiation feature for iosrtc. I’ve thought that an interesting option could be providing default CSS controls with the plugin (like the HTML5 video tag does) and allow the users to enable/customise them via a JSON object and overriding CSS properties… something like

{
  "enableDefaultControls": true,
  "controls": {
    "startCall": true,
    "stopCall": true,
    "holdCall": true,
    "fullScreen": true,
    "switchCamera": false,
    "mute": true,
    etc...
  }
}

Then provide default CSS classes (.stopCallButton, .startCallButton, .fullScreen, .fullScreenButton, etc) for both iOS and Android (xwalk):

SCSS example:

.fullScreenButton {
    position: absolute; 
    bottom: 0;
    float: right;

    /* PNG Icon */
    background-image: url('path/to/fullScreenButton.png');
    background-repeat: no-repeat;
    background-position: 25px 0px;

    /* Scalable Vector Icon */
    i {
        font-size: 25px;
        color: white;
        vertical-align: middle;
    }

    /* Platform Specifics */
    .platform-android { 
        // Android Stuff
    }

    .platform-ios { 
        // iOS Stuff
    }

    /*  Modify appearance depending on the resolution */
    @media (max-width: $screen-sm-max) {
        display: none;
        etc…
    }
}

The defaults could result in a badass UI like:

This way the users would have a straightforward out-of-the-box solution to use controls on top of the video, and also a starting point with simple logic in case they want to go further customising their UI’s.

Just an idea...

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

I’ve thought that an interesting option could be providing default CSS controls with the plugin (like the HTML5 video tag does) and allow the users to enable/customise them via a JSON object and overriding CSS properties… [...]
Then provide default CSS classes (.stopCallButton, .startCallButton, .fullScreen, .fullScreenButton, etc) for both iOS and Android

There are no start/stop/hold/fullscreen "buttons" in the WebRTC spec, and there shouldn't be. That assumes "softphone vision" and WebRTC is beyond that.

I agree that rear video UIView is useful, but just that. It's up to the app/developer to place "control" buttons on top of the video view when needed.

Regarding the usage of CSS classes to tell the plugin whether to put the video UIView in front/rear, I don't like that approach. CSS classes are up to the application and they are supposed to provide custom style/design to the app. I'd rather custom data-xxxxx attributes as suggested above, as that is exactly the purpose of data-xxxx attributes: to add custom properties to the HTML element. Or the negative z-index...

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Said that, I will implement the feature this morning XD

from cordova-plugin-iosrtc.

telemakhos commented on May 13, 2024

There are no start/stop/hold/fullscreen "buttons" in the WebRTC spec, and there shouldn't be. That assumes "softphone vision" and WebRTC is beyond that.

Yeah I know, I was just trying to get around the UI problem and the question of placing HTML on top. My suggestion was just an idea for convenience.

Said that, I will implement the feature this morning XD

Fair enough... 😁

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

Hi, first thanks for the wonderful work @ibc (and eface2face for supporting it), where's the BUY_ME_BEER.md in the repo? ;)

To shime on this issue (if I may):

The "rear" video UIView was the first thing I personally looked for when reading through the docs yesterday. A gazillion +1 from me on this one I believe it's a fundemental thing to have.
I feel everything-CSS that is otherwise for the purpose of demonstration (and you already have a an external demo repo) should remain outside the scope of this project. Sorry @telemakhos, nice screenshot though :)
I personally perfer data-xxxx, but that's just taste imho. If you already rely on z-index:-1 kind-of-thingie then might just as well stick to that, both approaches are equally fine, as long as the functionality is there :)

Gload to see someone did actually nail this project for a change :)

from cordova-plugin-iosrtc.

telemakhos commented on May 13, 2024

I feel everything-CSS that is otherwise for the purpose of demonstration (and you already have a an external demo repo) should remain outside the scope of this project. Sorry @telemakhos, nice screenshot though :)

I didn't know about the demo app!... I guess that would be definitely a better option for the reference client with controls that I was suggesting.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

https://github.com/eface2face/iOSRTCApp :)

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Guys, I tried yesterday with zero success. Don't know why but I was not be able to place the video UIView behind the web view and make the web view transparent. I also had issues when using the zPosition attribute of the layer video layer.

I will try it again next week.

from cordova-plugin-iosrtc.

telemakhos commented on May 13, 2024

No worries. Are you trying to replicate the gmaps technique stacking the subviews programatically or are you trying to come up with a custom solution?

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Let's say I must expend more time looking into phonegap-googlemaps-plugin...

from cordova-plugin-iosrtc.

RangerMauve commented on May 13, 2024

This would be super useful. Right now I've had to manually hide the video view when using the phonertc plugin whenever I wanted to render anything over the video.

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

It would be really useful, but my time is limited. PR are welcome.

from cordova-plugin-iosrtc.

oNaiPs commented on May 13, 2024

@ibc I could try to help on that issue. This is one of the few issues I have to fix before using your project. Did you actually create a branch and started working on it?

Thanks

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

I started working on it with no success. I will upload a branch on Monday and notify it here.
Thanks a lot.

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

I've upload a new branch "rear-video" to implement this feature. For now it has the same content as master since nothing that I tried did work so I've removed it (it was useless).

Notes and problems I remember:

My idea is that a negative z-index makes the video element UIView to be placed behind the HTML view (the Cordova UIView).
Stuff must be done in src/MediaStreamRenderer.swift.
Note that I use two UIView:
- elementView: It has same size as its corresponding HTML <video> element.
- videoView: It is placed over the elementView. Its purpose is to have a different size (which is cropped by the parent view) so CSS object-fit can be implemented (already done).
So we must work on the elementView leaving the videoView untouched.
In my attempts I was not able to place the elementView behind the Cordova view (it became invisible even if the HTML has no background color, not sure if something else must be done in the HTML UIView).
I expect issues due the usage of self.elementView.layer.zPosition = CGFloat(zIndex) since it seems that zPosition may alter how different layers are rendered. Currently if a negative z-index is set for the video element, the elementView just disappears.

Hope somebody can help on this :)
If so, please fork the "rear-video" branch and let work on it.

from cordova-plugin-iosrtc.

yocontra commented on May 13, 2024

This is the last outstanding issue preventing us from being able to use this in production. If you need anything at all, I'm down to help.

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

How to help is explained in my previous comment.

from cordova-plugin-iosrtc.

derMani commented on May 13, 2024

Hi IBC, in previous projects, we implemented an RTMP (flash) Video component for iOS as a native video component (without Cordova though), which runs in the background of a WebView component.
With transparent body settings it was possible to use it in the background and it worked pretty well vor us.

We want to move away from flash now by using WebRTC for future projects. I don't know if I can help you with this, but all we had to do in our project was this:

- (void)viewDidLoad
{
  /*** other stuff ***/ 

// set WebView Transparent
 webView.opaque = NO;
 webView.backgroundColor = [UIColor clearColor];
}

In the HTML, i used

body {
  background-color: transparent
}

If you knew this already, sorry for bothering you ;) I just came along this thread and thought this could help

from cordova-plugin-iosrtc.

makc commented on May 13, 2024

maybe a bit of off-topic, but can video with src = URL.createObjectURL(stream) be used with gl.texImage2D as here? seeing all these UIView refs I guess it can't, correct?

edit: #65

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

@derMani thanks a lot for pointing that out. I will check it soon.

@makc sorry, I don't know what that is about. Anyhow please feel free to comment about it in another issue.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@ibc Did you ever get the chance to look into derMani's workaround?

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Not yet, sorry.

from cordova-plugin-iosrtc.

acylum commented on May 13, 2024

Maybe this can work: Include a local HTTP server in the webRTC plugin to serve the video content to the video tag - this can hopefully allow both overlayed HTML elements along with full CSS styling control. We used this technique when we created an RTMP plugin...

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

@acylum That would produce a really uncomfortable delay. Video player in browsers (including Safari) do cache.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

This is in the works, so far the video successfully playing the background (HTML body must have transparent background), z-index support with negative values will be completed this week so that multiple videos (UIViews) are stacked on top of each other in order. Anyone that wishes to help testing this prior to sending a PR please feel free to PM me.

from cordova-plugin-iosrtc.

acylum commented on May 13, 2024

Sweet! I'd like to test it. Currently, my work around is to launch a wizviewmanager child window over the main view - so basically the webrtc video is "sandwiched" in-between. You have to use the window messaging to allow button interaction from wizview back to your main view.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@acylum awesome thanks! just replied you :)

from cordova-plugin-iosrtc.

jkuri commented on May 13, 2024

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

+1 are good, but pull requests are even better :)

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@ibc I have this working in my fork, see my last commit here: https://github.com/1N50MN14/cordova-plugin-iosrtc/commit/30ccdaa583b1f33f790bb8db3a5789fcdd8088e4 I'm not able to send a PR just yet, but feel free to take it for a spin in the meantime

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Great! Let me handle that on next week.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

Take your time

from cordova-plugin-iosrtc.

derMani commented on May 13, 2024

Hey guys, just wanted to know if you have an update on this.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@derMani I submitted a PR for this one, waiting for ibc to look into it and eventually approve it, otherwise you can use my fork for now.

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Guys, I won't be able to merge the PR until the second week of the new year.

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@ibc No worries, Merry Christmas! ;)

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Same! :)

from cordova-plugin-iosrtc.

derMani commented on May 13, 2024

Happy new year everyone! @1N50MN14 your PR works superb 👍 Thx :)

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@derMani Du bist willkommen und Frohes Neues Jahr :)

from cordova-plugin-iosrtc.

telemakhos commented on May 13, 2024

Glad to see this is finally coming to fruition. Great job @1N50MN14 . I guess the next goal is supporting the trick on both UIWebView and the new WKWebView... see #90

from cordova-plugin-iosrtc.

1N50MN14 commented on May 13, 2024

@telemakhos Yup I can make that work once @saghul submits his PR for WKWebView support

from cordova-plugin-iosrtc.

bassemBMI commented on May 13, 2024

I used @1N50MN14 branch but the video doesn't work.
@ibc there is any fix planned for this version ?

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Hi, I'm sorry I couldn't properly check the status of the PR #98. I commented some issues regarding the PR (for example: /lib/libWebRTC-LATEST-Universal-Release.a can NOT be removed from the repo) and I'm not sure whether that PR handles all the required use cases without breaking working code (in fact I've read some users reporting wrong behavior).

So, if somebody can spend time to test #98 and provide proper conclusions I will merge it.

from cordova-plugin-iosrtc.

xpluscal commented on May 13, 2024

Sorry to bother you again @ibc, but are there any news regarding this issue?

from cordova-plugin-iosrtc.

bitjson commented on May 13, 2024

Not directly related to this repo, but a plugin I work on just released support for rendering the camera preview layer behind a transparent UIWebView or WKWebView: bitpay/cordova-plugin-qrscanner

It's just a simple plugin, but might help if someone is trying to implement the same thing here: https://github.com/bitpay/cordova-plugin-qrscanner/blob/master/src/ios/QRScanner.swift

from cordova-plugin-iosrtc.

bassemBMI commented on May 13, 2024

Hi @ibc I join m'y voici to @xpluscal in order to have an idea when it can be fixed.
Thx.

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

I think the status regarding this PR is already explained in comments above, but I'll try to summarize here:

The author of the PR has already said that he's working in his own project so it's expected that he's no longer maintaining the code in this PR.
There are also comments from some users that tested the PR and they reported some issues in some cases.
I don't have time to test all the required use-cases nor to look at how other projects implement a similar feature.

So, if someone wants to contribute, take this PR, apply it into a fork of the official master branch of the plugin, check all the use-cases and document them, then I'll be very happy to accept a new PR.

from cordova-plugin-iosrtc.

derMani commented on May 13, 2024

Just upgraded to 3.0.0 of the Plugin and the PR still works for me.

The steps are basically the same as before (and suggested in the PR)

Hide Video Controls for video element in Javascript
Edit PluginMediaStreamRenderer.swift to make sure the webview is transparent.

I think I could provide a small example project, if you wish.

@ibc What exactly are "all use cases" ? :-)

I just created a web page, made sure everything had the correct CSS and after that I was able to see the video through my transparent elements.

Maybe that's the problem for some people here, you have to make everything transparent on your page, if you want to look through it. It's easy to forget about a few elements, which might still have a background-color (like body, html, ...)

from cordova-plugin-iosrtc.

chris087 commented on May 13, 2024

Actually is not working for me.

Using 3.0.0;

Having all transparant. (verified several times).

body {
background-color: transparent;
}
html {
background-color: transparent;
}
.pane, .view {
background-color: transparent;
}

[nav-view-transition="ios"][nav-view-direction="forward"], [nav-view-transition="ios"][nav-view-direction="back"] {
background-color: transparent;
}

#myMiniVideo {

width:30%;
height:30%;
position:absolute;
right:0;
bottom:0;
z-index:0;

}

#videoscreen .video_controls {

position:absolute;
right:50px;
bottom:50px;
z-index:15;

}

Video overlays the .video_controls... :(
no matter what z-index the myMiniVideo have. When setting -1 z-index to myMiniVideo, the video is not visible.

Changes in PluginMediaStreamRenderer.swift :

// The browser HTML view.
self.webView = webView
self.eventListener = eventListener
self.webView.opaque = false
self.webView.backgroundColor = UIColor.clearColor()
// The video element view.
self.elementView = UIView()
// The effective video view in which the the video stream is shown.
// It's placed over the elementView.
self.videoView = RTCEAGLVideoView()
    self.webView.superview?.addSubview(self.elementView)

    self.elementView.userInteractionEnabled = false
    self.elementView.hidden = true
    self.elementView.backgroundColor = UIColor.clearColor()
    self.elementView.addSubview(self.videoView)
    self.elementView.layer.masksToBounds = true

    self.videoView.userInteractionEnabled = false

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

@derMani

What exactly are "all use cases" ? :-)
I just created a web page, made sure everything had the correct CSS and after that I was able to see the video through my transparent elements.

I mean how it works. Which values do z-index accept and what happens if z-index is negative. And what happens if a <video> is placed on top of another <video> with different z-index, etc.

To be clear: I don't have an iOS device so I won't check all the cases, that's why I need complete documentation regarding how this PR works.

BTW as you can see, a comment after your says that it fails. If I accept this PR as it is, I'll become responsible of such a problem.

from cordova-plugin-iosrtc.

ibc commented on May 13, 2024

Done in #179

from cordova-plugin-iosrtc.

cozzbie commented on May 13, 2024

Is there anyway to redraw the rendering video onto a HTML5 canvas?

from cordova-plugin-iosrtc.

Drawing HTML on top of a UIView is possible about cordova-plugin-iosrtc HOT 60 CLOSED

Comments (60)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent