Taking snapshots with WebChimera Plugin

[TeeJiiAl]

Taking snapshots with WebChimera Plugin

[RSATom]

Thank you for request.

Do you need it within QML, JS or both?

[jaruba]

@RSATom Are you planning to add this to the QML, JS API?

I have been asked before by another user if the player can take screenshots.

[RSATom]

It's possible for both, I just don't sure what picture format use for JS side...

[RSATom]

http://doc.qt.io/qt-5/qquickitem.html#grabToImage - it's for C++ side, I will just need add some wrapper for it...

[jaruba]

I just don't sure what picture format use for JS side...

Use PNG if you can, it has the best compression.

[TeeJiiAl]

Hello, JS is preferred. We have no experience with QML, but if it works better with that we will look into that.

I also think PNG would be good idea.

[RSATom]

@jaruba I think it's possible.

@TeeJiiAl, How do you plan to use this picture? I should know it to select most suitable implementation.

[jaruba]

@RSATom Couldn't you send it back as string encoded in base64?

That way it could be saved to a file or even displayed directly from the web page without needing to save it anywhere. But the function to call the screenshot should be async in this case, to not freeze the page until the process is finished.

[RSATom]

@jaruba, base64 - yep, without any problem I think.

[TeeJiiAl]

The player would be part of the metadata management system, which provides acces to VOD server. Files are in H.264 format wrapped in TS. Files are streamed to clients. Snapshots would be used capturing end titles of the programs for cataloguing purposes. Right now we use web plugin provided in VLC 2.1.5.

[RSATom]

@jaruba, Does it has any meaning within QML? Cite from http://doc.qt.io/qt-5/qquickitem.html#grabToImage:

Note: This function will render the item to an offscreen surface and copy that surface from the GPU's memory into the CPU's memory, which can be quite costly. For "live" preview, use layers or ShaderEffectSource.

i.e. layers and ShaderEffectSource could be used for something similar (but don't sure how exactly yet)

[RSATom]

Snapshots would be used capturing end titles of the programs for cataloguing purposes.

i.e. you plan send it to server, right? So base64 encoded png will go good enough for you, right?

[jaruba]

@RSATom I'm not sure if there is any good reason to have it in QML ftm.

I'm thinking it would be nice to have a "Screenshot" option in the context menu of the player, but even if it would have this option, in all circumstances it would still need to send the image back to JS in order to either save it to the disk (node-webkit) or prompt the visitor to download it (web use). So it could be enough to have it only in JS.

[TeeJiiAl]

They would be stored in local workstations for now, but maybe in later phase possibilty to store them in MMS. As long as you can read text from the image (end titles), it is fine.

[RSATom]

Ok, then I think it will be easy enough (I hope). Scheduled to next release.

[TeeJiiAl]

Great!

[RSATom]

Btw, what OS do you use?

[TeeJiiAl]

Mostly Windows 7, few use OS X.

[RSATom]

It's good, since I will start implement it on Windows first.

[appollosputnik]

Is this features grabImage implemented Dear Jaruba? If yes then please tell me in which version & how to build it if it is differently needs to be built.?

[jaruba]

@appollosputnik

I can't implement it, it needs to be available in WebChimera Plugin, so @RSATom will need to make it when he finds some time.

But I see this feature is pretty famous though, someone asked me about it even before this request was ever created.

[RSATom]

This task is still on the begining of my todo list, but unforunatly I'm very limited in free time...

[appollosputnik]

@RSATom May I know approx by when you're planning to implement in plugin? I need it urgently so if you could give me some hints if I can try myself to implement it it will be highly appreciated. Can you do that ? How I should proceed about it.

[RSATom]

To solve this task we require function which will convert QmlVlcI420Frame to QImage, or at least RGB array. I'm already solved this task with shaders: https://github.com/RSATom/QmlVlc/blob/master/SGVlcVideoNode.cpp#L65, but in this case the same should be done on CPU with C/C++.

[appollosputnik]

@RSATom - O so good RSATom. Now I think Jaruba can take it to do it in CPU?
@jaruba - Do you think dear Jaruba that you can include it in Plugin & fetch it to CPU?

Please update me sir.

[RSATom]

@appollosputnik, you didn't understand. It's not implemented yet. I just tell how it should be done.

[RSATom]

Implementation on shaders could not be used in this case, so it should be reimplemented without shaders.

[RSATom]

@appollosputnik, do you have some deadline?

[appollosputnik]

OIC. Yes @RSATom I have deadline till this week. Is it possible to do it by the end of this week. I will really appreciate it.

I am really very happy with other features & you guys support. You guys are into fantastic project doing. I would like to make some donation on this project to grow it bigger. I will always be in sync with this project. Please give me your official PayPal ID where I transfer my donation.

Please try if you can implement it by the end of this week.? Let me know if not how long time it may take?

Regards
Sujan

[RSATom]

@appollosputnik, you could see donate button on http://www.webchimera.org/ or use my email (RSATom at gmail dot com) as PayPal ID

Is it ok if it will be ready to monday (May 25)?

[appollosputnik]

Ok I will do it using either.

Yes I think I will be OK by Monday if you can make it. I will extend my deadline a bit. It will be perfectly Ok. Please start with it as per your schedule. And post a message when it's done with the guidelines of usage. I thanks you both @RSATom and @jaruba for your support, it's highly appreciated.

Regards
Sujan

[RSATom]

Ok, Maybe I will do it earlier, just don't want promise, since I have little bit more free time on weekends.

[appollosputnik]

Ok dude take your time Update me when I can use it.

Regards
Sujan

[RSATom]

test version: https://dl.dropboxusercontent.com/u/94817236/WebChimera_0.2.7.2_vlc_2.1.5.msi
API changes (will add it to wiki on release):
Qml: added global function takeQmlItemShot( shotId, qmlItem )
JS: added event QmlItemShot( shotId, qmlItemShot ). qmlItemShot will be base64 encoded png.

Shot could be taken from any visual Qml item. If you need take shot from video - you should pass id of VlcVideoSurface.

[jaruba]

@RSATom Great job! I will start testing it tomorrow.

I wonder if it will work to take a screenshot of VlcVideoSurface while it is scalled. (I guess it should)

[appollosputnik]

@RSATom Excellent job. I am checking it now.
Thanks a lot sir.

Regards
Sujan

[RSATom]

@jaruba, it should work with any visual item

[appollosputnik]

@RSATom I have created a button on toolbar like this
// Start SnapShotNext Button
Loader.ToolbarButton {
id: snapBut
icon: "Snap"
iconSize: fullscreen ? 8 : 7
anchors.leftMargin: 1
glow: ui.settings.buttonGlow
onButtonClicked: buttons.clicked("snapshot");
}

And in Buttons.qml I added this code:
if (action == "snapshot") if (typeof settings.preventClicked[action] === "undefined") {
QmlItemShot( VlcVideoSurface, base64)
}

This should work right in version 0.2.7.2 ? But it's not working.

1st of all I am not able to see any video in version 0.2.7.2, only sound is coming.

[RSATom]

You use it wrong. Please wait while @jaruba add shot support to player, I think he will do it soon.

[jaruba]

@RSATom Please check gitter chat, there is a bug with WebChimera Plugin v0.2.7.2.

[jaruba]

Also, please confirm that I understand the usage of the new function/event.

takeQmlItemShot( shotId, qmlItem )

Should be used in QML in order to make a screenshot. qmlItem will be the id of the QML component that will be captured in the image and shotId is simply an integer that will help identify the newly created image. (as creating the image is async)

When the image has been constructed, then:

QmlItemShot( shotId, qmlItemShot )

will be triggered in JavaScript, shotId will be used in order to identify the created image with the selected integer value that was set in takeQmlItemShot() and qmlItemShot will be a data:image/png;base64 encoded string.

Is this presumption correct?

[RSATom]

Yes, it's correct

[RSATom]

But if you try take next shot before QmlItemShot arrives - previous shot will be lost.

[RSATom]

Btw, maybe you have some suggestions about this API?

[appollosputnik]

@RSATom Seems to be perfectly fine. Just one suggestion is it too difficult to give an option for encoding in JPEG format also. Many users prefer JPEG. If it is not too difficult please try to include in the option of saving the image.

[jaruba]

@appollosputnik

The screenshot should not be JPEG, it is best for it to be PNG. The reason for this is that PNG has a lot better compression.

Images that have 3MB in JPEG format might only have 700kb (or maybe even less) in PNG format and there is absolutely no visual difference between the images. PNG will download A LOT FASTER to the user's PC and use less bandwidth from your server. There are just to many advantages to using PNG and it is always recommended to use PNG over JPEG in all web based projects.

[jaruba]

Btw, maybe you have some suggestions about this API?

But if you try take next shot before QmlItemShot arrives - previous shot will be lost.

Then why even add a shotId if it cannot support multiple parallel instances of taking screenshots? I don't see any reason for it.. Just makes it more complicated to explain in the API docs.

[RSATom]

@appollosputnik, I'm agree with @jaruba, png is better than jpeg by many reasons.

[RSATom]

Then why even add a shotId if it cannot support multiple parallel instances of taking screenshots? I don't see any reason for it.

The reason is simple, taking shot started on QML side, but result arrives at JS side - so it's possible shot will require some additional info to handle it right way on JS side.

But I have idea how I should change shot API. Maybe it was wrong idea init operation and get result on different sides (QML vs JS). Maybe we should have takeQmlItemShot( qmlItem ) function and qmlItemShot( qmlItemShot ) event both on QML side. And after that it will be developer decision if it required send shot to JS side with QmlStringMessage for example.

[jaruba]

@RSATom makes sense, as QML is on the client side it shouldn't add any significant delay, just gives more programmatic control over the process.

With just takeQmlItemShot( qmlItem ) and qmlItemShot( qmlItemShot ) in QML, I can easily make something like .takeScreenshot( [bool] ) function and .catchScreenshot( image ) event in WebChimera Player's JavaScript API.

And [bool] from .takeScreenshot( [bool] ) can be true if you want to prompt for image download directly without listening to .catchScreenshot( image ) event, or false to not do anything automatically and then would need to listen for the event for further processing.

[appollosputnik]

@RSATom Ok, that will be fine. we will work with PNG that will be better.

Thanks a lot for all your dear @RSATom and @jaruba

[RSATom]

@jaruba, ok, working on it right now.

[appollosputnik]

@RSATom @jaruba Ok dudes take your time.

[RSATom]

New test version: https://dl.dropboxusercontent.com/u/94817236/WebChimera_0.2.7.3_vlc_2.1.5.msi

Shot API was changed. Now all related members are on QML side:

• takeShot( qmlItem ) - global function
• shotReady( shot ) - global signal

Usage example: https://gist.github.com/RSATom/11d39328069282ff8f65

[appollosputnik]

@RSATom Dear please give an option of saving it in JPEG format. I need it it's very important for me. If not possible tell me, I will have to convert it from html if possible at run time. But I need it either in plugin or player. Please let me know if you can help me in that.

I am testing the latest version, thanks a lot for the fix.

Regards
Sujan

[jaruba]

@appollosputnik

You can convert a data:image/png;base64 to data:image/jpeg;base64 with JavaScript as described in this answer:

http://stackoverflow.com/questions/17590136/base-64-png-string-to-base-64-jpg-using-java-or-javascript#answer-19286473

But I'm not planning to add an entire library to encode jpg files in the Player just for this feature as it can be done outside the Player and Plugin anyway..

[appollosputnik]

@jaruba that will be great, Jaruba, is it possible for you make an API for JPEG after capturing the actual PNG image, in webchimera.js file?

Then I will call that in Buttons.qml on click of some button on the toolbar panel.

Regards

[jaruba]

@appollosputnik

I will not add it to webchimera.js, but I can probably show you how to do this correctly with an example page. Also, do you need the screenshot button to be in the toolbar?

I would personally add it in the context menu (from right click) and as a hotkey. Would that be ok for your needs?

[appollosputnik]

@jaruba Ok Jaruba please give me guidance how I can do it dynamically in my player.html file, that should be fine.

Context menu will create problem Jaruba, because while taking shot if the menu appears that may also appear in the picture right?

Better option would be a dynamic button on left or right tool bar panel. That should be fine.

Regards
Sujan

[jaruba]

@appollosputnik The screenshot is not made of the entire player, we can choose exactly what component will be extracted from the player and added in the image. In this case, we will only screenshot the video feed, none of the user interface items, so the toolbar, context menu, etc. will not appear in the picture.

So is it alright then to add it to the context menu and as a hotkey (CTRL + S)? Personally I don't want to add it to the toolbar, there are already a lot of buttons in the toolbar and I still want to leave some space there for people that want to add their own custom buttons.

If I add it to the context menu and the hotkey then it will always be available for you in the same way in all future versions of WebChimera Player too, so you can update easily to newer versions of the Player later on if you want.

[appollosputnik]

@jaruba OK dear Jaruba it'll be fine I think. Then add it to the context menu with a hot key. That will be great.
Thanks

[appollosputnik]

@jaruba & @RSATom Please try to release the plugin & player by this weekend if possible. But it will be fine if you do that by coming Monday also.

Thanks a lot.

Regards
Sujan

[RSATom]

@appollosputnik, how about donation/bounty? Now all depends from @jaruba, so you could create issue at WebChimera Player repo and place bounty on it.

[RSATom]

@jaruba, maybe it worth change function/signal names to takeSnapshot( qmlItem ) and snapshotReady( snapshot ) ? What do you think? it seems shot is wrong word for this feature :)

[appollosputnik]

@RSATom Dear Sergey I will make surely some donation by the end of next week. Don't worry about that at all. I will be with you guys for long term.

@jaruba Dear Alexandru please make a release of the complete version with the source so that I can start using it.

I really appreciate you guy's good work, I am with you for long term don't worry. We will help each other.

Regards
Sujan

[jaruba]

@RSATom It's not really necessary, but these lines in your demo:

onTriggered: takeShot( videoOutput );
target: plugin;
onShotReady: fireQmlStringMessage( 0, shot );

Are really funny when you first read them.. :)))

[RSATom]

yea, it's funny :) anyway I think I will change it. so please don't release new Player version before I release plugin. If you already tested it, I even could do it today.

[jaruba]

I wonder if it will work to take a screenshot of VlcVideoSurface while it is scalled. (I guess it should)

@RSATom It does not take a screenshot of the scalled image, it screenshots the entire image as if it wore not scalled.

@appollosputnik I finished the screenshot feature in the player, you can download it from:
http://webchimera.org/samples/WebChimeraPlayer-screenshot.zip

It converts the image to jpeg automatically before prompting for download. I also added a Digital Zoom demo to the package.

But I'm guessing you need it to screenshot the video exactly where you zoomed in on the video? If so it will not do that correctly as I've mentioned above it does not take into consideration that the video is zoomed in..

It's not impossible to make it screenshot the zoomed image.. but it's not easy to do either.. I really hope that you do not need it to screenshot with the zoom feature.. Otherwise it will take some serious coding to make it do that..

[RSATom]

@jaruba, did you make zoom by place VlcVideoSurface into Rectangle with crop=true?

[jaruba]

@RSATom no, I'm using .scale.origin.x, .scale.origin.y, xScale and yScale. Scale QML Type

It's a Transform effect. The screenshot feature does not seem to take into consideration any transformation effect.

It would still be possible to make the screenshot see this difference in the player.. If I write the base64 png image to a canvas then normalize the canvas with the player by cropping and resizing it then pulling the image from the canvas to prompt for download.

But I can't even predict the amount of time it would take to do this correctly.. There are so many problems that can appear in the normalization process..

[appollosputnik]

@jaruba Thanks Dear Alexandru I am testing it right now.

@RSATom Dear Sergey can you give me the source of Plugin version 0.2.7.3? Then I can use it according to my need.

Regards
Sujan

[appollosputnik]

@jaruba I tested it Jaruba it's awesome, fantastic, you guys made it so beautiful. Thanks a lot. I will surely make some good donation, please wait till next week. Keep up the good work we will build bright future with Chimera.

@RSATom Dear Sergey it will be great if you could give me the source of Plugin version 0.2.7.3 soonest. I can proceed forward then.

Keep it up..Thanks
Regards
Sujan

[RSATom]

@jaruba, you could try create second VlcVideoSurface and place it into Rectangle with crop=true. After that pass id of rectangle to takeShot - in this case you should get only visible part of video frame.

[RSATom]

@appollosputnik, current master branch = version 0.2.7.3

[appollosputnik]

@RSATom master branch is still showing version 2.7.1 Dear can you post the link ?

[RSATom]

I didn't tag 0.2.7.3, so just look to the latest commit at the master branch. Btw, I will release v0.2.8 within a hour, so you could wait a little to take release version - but you will have to change qml since I've changed snapshot function and signal names a little.

[RSATom]

I've just released WebChimera v0.2.8 (only for Windows).
Important:

• takeShot( qmlItem ) function was renamed to takeSnapshot( qmlItem )
• shotReady( shot ) signal was renamed to snapshotReady( snapshot )

updated usage example: https://gist.github.com/RSATom/ccc002c35e8fe1303306

@jaruba, please update function/signal/signal argument names in Player

[appollosputnik]

@RSATom Thanks a lot dear Sergey.

@jaruba Please update me Alexandru which version of player shall I use? I think the one above you posted is more active right compatible with 0.2.8?

Please share the compatible version of player.

Regards
Sujan

[RSATom]

@appollosputnik, you could fix it yourself, just change function, signal and signal argument names to mentioned above. Overall logic was not changed there.

[appollosputnik]

@RSATom Ok dude. Let me do that. Please give me the link of VLC version 2.2.1 for Chimera compilation which I need to keep src/Win/Wix folder. I am not able to get it. My build is failing.

[RSATom]

http://get.videolan.org/vlc/2.2.1/win32/vlc-2.2.1-win32.7z

[appollosputnik]

@RSATom Thanks a lot dude.

[appollosputnik]

@RSATom @jaruba There is no 'shotReady' in player it is 'onShotReady'

'onShotReady' should be changed to 'snapshotReady' right ?

[jaruba]

@appollosputnik

http://webchimera.org/samples/WebChimeraPlayer-screenshot.zip

This new package now supports WebChimera Plugin v0.2.8, still does not support making a screenshot when it's zoomed though.. it will take a while to make it work with zoom..

[appollosputnik]

@jaruba Thanks a lot dude, let me try it.

Regards
Sujan

[jaruba]

@appollosputnik Is taking screenshots when the Player is in zoomed mode important for your project? Because I would delay it for a bit if it is not crucial.

@RSATom Is this going to be implemented soon on Mac / Linux? Or should I check for platform for this feature before committing to master?

[RSATom]

It already should work on all platforms, I just need find some time to build mac version.

[RSATom]

so you could just check v.0.2.8 if you wish.

[jaruba]

@RSATom Btw, I managed to make a good converter for base64 images in JavaScript by inserting the image into the page then drawing the element on a canvas and exporting the canvas to a base64 string again and choosing any image format I want.

Thinking of making a github repo with this solution as I haven't really found any easy way to do it while checking google. :)

[RSATom]

do you mean canvas can accept base64 encoded images directly?

[jaruba]

Not directly, I put base64 encoded images in a <img> tag first, then I use .drawImage() to draw the <img> tag on the canvas after which I use .toDataUrl() to convert the canvas to a base64 encoded image.

So I can actually give the option of selecting what image type you want in the API.

[RSATom]

now understand, thanks for explain.

[appollosputnik]

@jaruba Dear Jaruba taking screenshot when zoomed is also important dude. Please try if you can do it , I can understand your time is too hectic but without that feature will not take complete shaped dude. Please take some time try to include it in player.

Is it the work to be done in player or plugin.?

[jaruba]

I'm not sure if it can be done in the plugin, but in the player it is very complicated from my point of view. Scale uses totally different parameters then those needed for crop: true, I actually tried to do it with crop: true in the past and failed terribly.

Scale actually saved me as it worked a lot better and it was a lot easier to use for this feature. Needing to do it all again with crop: true.. I'm not even sure how long it will take..

[appollosputnik]

@jaruba You're right. I am not sure whether it will be easier in plugin or player. But the viewing volume has to be scaled equally with respect to zoom before capturing the container. It will be some what difficult I understand. But you can best judge.

Just try if it could be done shortly or take some time as long as you need, I will try to adjust without this for time being. But remember this features makes it complete in full shape.

Regards
Sujan

[RSATom]

@jaruba, is there any online demo available with zoom feature?

[jaruba]

@RSATom only here atm: http://webchimera.org/samples/WebChimeraPlayer-screenshot/digital_zoom.html

[RSATom]

Thanks, it's enough.

[jaruba]

package that includes this demo here: http://webchimera.org/samples/WebChimeraPlayer-screenshot.zip

[RSATom]

Thank you, Alexandru.