Portrait mode does not work ?

[yuensunn]

Portrait mode does not work ?

[xavierjs]

Take a look here : https://developer.mozilla.org/en-US/docs/Web/API/Media_Streams_API/Constraints
idealWidth and ideal height are only suggested values : it impacts the resolution of the video. The resolution of a video is different from its displayed size. If the webcam cannot set to the given values it can return a video with a different resolution.

Set the displayed size by setting the width and height attributes of the canvas element. Then the video will be cropped to fit into the canvas. Give the idealWidth and height as if it was always lanscape values.

If you want to switch dynamically between landscape and portrait mode, you need to resize the canvas, call the JEEFACEFILTERAPI.resize() function to recompute the crop params of the video and to recompute your projection matrix. You should take a look at the GLTF FULLSCREEN DEMO : https://jeeliz.com/demos/faceFilter/demos/threejs/gltf_fullScreen/

[yuensunn]

Please correct me if I'm wrong. Based on what you have just said, it seems like there is no way to have a true portrait video instead of cropping it from a landscape video? I have tried this sample demo on mobile, the video itself seems like it was cropped out of a landscape video (way too close up even with my arm extended at full length) sample image

[xavierjs]

If the camera is in landscape mode (a phone handled horizontally or a laptop), you will get a landscape video from getUserMedia API even if you request portrait ideal width and height.

Same for portrait mode : if your camera is in portrait mode, you will get a portrait video.

Then the video is cropped to fit the canvas the same way than CSS background: cover does : if the canvas is more landscape than the video we crop only top and bottom margins of the video, otherwise we crop only left and right margins.

But depending on the camera configuration sometimes the default zoom is too high. There is a draft API to give access to the camera intrinsic parameters but this is not available yet.

There are some informations about this issue here : #22

The demo you have tried (https://jeeliz.com/demos/faceFilter/demos/threejs/gltf_fullScreen/) is fullscreen. So it is in a very portrait mode (if we compute the canvas aspect ratio ac = canvasWidth/canvasHeight, ac is small).

I think the video from the webcam is less portrait (it depends a lot on the phone brand, the OS, the camera maker...). If we compute the video aspect ratio, av=videoWidth/videoHeight, we have av>ac .

So we need to crop the left and right margins quite a lot to display the video in fullscreen in order to not distort it or display blank space at the top/bottom.

So what I suggest is to not use a fullscreen canvas. With a square canvas for example we have ac=1 and the video should be less cropped (try the THREE.JS boilerplate demos for example).

[yuensunn]

I guess I will just use a square canvas until the new API releases. Thanks for the prompt feedback.

[xavierjs]

This is not a problem with FaceFilter API, so new releases won't change anything. I just wish the WebRTC standard will be better applied by browser and hardware builders.

[xavierjs]

I close the issue. Do not mind to re-open it.

[ouzhou]

i tried your gltf_fullScreen demo, it works
but the number of times it lost face is more bigger than rect canvas
any performance loss when fullscreen?

[ouzhou]

@xavierjs sorry i dont have any screenshot or any test code, i just feel it become more worse than use rect canvas, so if width !== height really cause performance loss?

[xavierjs]

In fullscreen mode, there are more pixels so it requires more computationnal power and it may be slower. It depends on your graphic hardware. The performance depends on the area of the canvas.