auduno/headtrackr

Javascript library for headtracking via webcam and WebRTC/getUserMedia

headtrackr's Issues

Build from source didn't work

I guess it uses an old version of Grunt and there are no development dependencies mentioned in the package.json, although they are used in the Grunt task. I would like to do a pull request if it's convenient for you. This is a great project!

On the robustness of the headtrackr

Hi,

I am trying to use the headtrackr for getting ONLY the X position of the head.
I am using it in a small game I've build which looks pretty nice this far.

At the moment I ask my user to present his full face so the headtrackr can lock on his head and then I ask my user to tilt the screen a bit so that only the chin and up is visible(I want to hide the neck since low-cut tops create color interference).

I would like to ask 2 things:

What is the best advice you think I should give to my users in order to have my game as robust/accurate/fast as possible. What calibration recommendations do you suggest I should present and what are the ''perfect'' conditions for the headtrackr to work at it's best?

The goal is to make the head-tracking as robust as possible between different lighting environments.
Also I need to have the head-position detection as predictable as humanly possible(some times the headtracker goes all nuts on me and starts swinging right and left, losing it's center).

At the moment I only advice that during the head tracking, the user should ensure that he has his both sides of his face evenly and brightly illuminated.
Also as a second calibration step , I advice my user to tilt his laptop screen up until the point where the neck is not in the frame.

Second thing:

Of course any advice for any parameters I might pass on starting the headtrackr are welcome. (Should I use facetracking x position or headtracking x position?, Should I calculate angles etc etc)

Thanks in advance man and thanks for the work

Given a picture can I get the position of the eyes?

with any of clmtracker or with headtracker?
Thanks

Already found the way, thanks anyway!

canvas inputs

My project has a camera module already, so I wanted to us that canvas instead of the video. To headtrackr.js I added:

this.customStart = function(canvas, context) {
    canvasElement = canvas;
    canvasContext = context;

    // create ui if needed
    if (params.ui) {
        ui = new headtrackr.Ui();
    }

    // create smoother if enabled
    smoother = new headtrackr.Smoother(0.35, params.detectionInterval+15);

    this.initialized = true;
    run = false;

}
this.customTrack = function(canvas){
    videoElement = canvas;
    track();
}

and in my app:

// on receive canvas from other module
this._htracker.customTrack(canvas);

I also made some hacks around not sending an image before getting a response for the previous one. It could be useful to have a less-hacky way of bypassing the built-in getUserMedia handling.

Here is my demo: http://meemoo.org/iframework/#gist/5005786
Vid: http://vine.co/v/b6PntIHlaj6

facial feature tracking

Hi there. thanks for this cool library. i was wondering if it is possible to track facial features with this library. if not, could anyone suggest a way to get started? i know that i can get the haar cascades from opencv for eyes and other features and use those. i'm just not clear on how to actually integrate with this lib. thanks!

Stable Head Angle Value

Hello,
For a very large projects, i have to implement headtracking where the elements on the right & left side of the screen will become active if the user look at them(turn their head).

I implemented your code, but i have problem with returned event.angle.
The "event.angle" is not accurate. for example i can't say that:
if event.angle = 1.5 (user is looking at center of screen)
if event.angle > 1.6 (user is looking at the right side)
if event.angle < 1.4 (user is looking at the left side)

now sometimes event.angle return 1.5 when you are looking at the center of the screen and other times 1.7 etc. so i can't tell where the user is looking.

Do you have any solution for this please?

My code:

htracker = new headtrackr.Tracker({
        altVideo: {
            ogv: "./media/capture5.ogv",
            mp4: "./media/capture5.mp4"
        },
        calcAngles: true,
        ui: false,
        headPosition: false,
        debug: debugOverlay
    });
    htracker.init(videoInput, canvasInput);
    htracker.start();

document.addEventListener("facetrackingEvent", function (event) {

    if (event.angle < 1.58) {
        document.getElementById('female').style.color = "red";
        document.getElementById('male').style.color = "#5f6a72";
        console.log('left');

    } else if (event.angle > 1.62) {
        document.getElementById('female').style.color = "#5f6a72";
        document.getElementById('male').style.color = "red";
        console.log('right');

    } else {
        console.log('center');
    }

});

support for multiple faces

Is it possible to modify the library to support simultaneously tracking multiple faces?

xeyes 2.0 example

See http://cgestes.github.io/xeyes/ :)

Typo in code sample

Hi there,

on this page
http://auduno.github.io/headtrackr/documentation/reference.html

theres a " missing from the if(event.status ) block.

document.addEventListener('headtrackrStatus',
function (event) {
if (event.status == "getUserMedia) {
alert("getUserMedia is supported!");
}
}
);

Best,

Dylan

Firefox 18 support

The new version's of firefox ( 18 ) have webrtc support , It looks like headtrackr does not support that .

Does the Headtrackr works on images?

I want to detect a face on an image. Don't want to use clmtrackr because sometimes it does not converge or it takes to long.

Thanks!

Using Web Workers to Minimize Application Delay with Head Tracker

Hi All,
We are currently using the head tracker as part of a real time application which requires displaying time-sensitive images alongside head tracking. Although we are using timers to display these images, it seems to us that the head tracker execution is prioritized due to the single threaded nature of JavaScript, resulting in delays of fifty milliseconds more than the desired time interval for each image display. We cannot afford that kind of delay associated with timers so we are considering the use of web workers to help us achieve an illusion of multi threading and thus reduce these time delays. If anyone has used web workers to achieve a similar goal with head tracker, please let us know how it can be done and whether it has proved effective or not.
Thank you.

Overriding mediaStream Using altVideo

I have tried the altVideo parameter by pointing it to an arbitrary MP4 video. Currently its purpose seems to be to supply an alternative video with directions if webcam fails.

Our need is to run the headtracker as it would normally run with a mediaStream, but using a pre-recorded video where a person has captured her head movements. We need to do this to run some test cases on headtracker. Is this something that can be achieved using the altVideo parameter or another option in headtracker js?

multiple headtrackr

For line "track = function() {",

"var" is missing, the function becomes global and cause multi-headtrackr not working

Inputting data from headTracking event into an array

Hi,

Sorry if this is a stupid question, I am new to JavaScript and the like. I was wondering is there is a way to collect the data from an event (in this case I am trying to collect the z property from the headTracking event) and input it into an array?

I am trying to collect the data to work out the average of the heads z position over a given amount of time.

Let me know if anymore information is needed,

Cheers

Sean

targets demo does not work in modern browsers (chrome / brave / opera)

Hi, I'm trying to get the awesome targets demo to run - it's displaying the webGL components fine, but not responding to movement in front of the camera. Would it be possible to get a copy of the sourcecode for the targets demo ?

Thanks !

Z distance resets at some point

Hi,
I'm playing around with the z distance. If you get too close to the camera or too far away, the guessed z distance kind of resets itself and jumps to values like -150. Am I doing something wrong?

Here's my current example: https://vin-ni.github.io/z-to-em/

Thanks 👍

headtrackr not working on Chrome 42

http://auduno.github.io/headtrackr/examples/targets.html demo is not working on Chrome Version 42.0.2311.39 beta (64-bit)

Here is what the console is printing at the beginning:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.39 Safari/537.36 | WebGL 1.0 (OpenGL ES 2.0 Chromium) | WebKit | WebKit WebGL | WebGL GLSL ES 1.0 (OpenGL ES GLSL ES 1.0 Chromium)

In addition to a lot of warnings here is the error I get:
Could not initialise shader
VALIDATE_STATUS: false, gl error [1282]

The demo is running well on Firefox 36.0.1

Head Tracking Demo No Longer Available

hi,

Was hoping to reference your implementation of Facekat, using a different tracking method. The demo itself seems to be down:
http://www.shinydemos.com/facekat/

Any chance I could get at the source code or a link to a new url if it's just been moved.

Awesome work and good blog write up as well!
Francois

No possibility to stop mediastream opened by headtrackr.init()

It is not possible to close the getUserMedia mediastream opened by headtrackr on init(), since the mediastream pointer is discarded after initialization. The solution is to not discard the pointer and possibly add a method to close/stop the stream.

Movember Games

Hey Mate,

I couldnt find your personal email to write this directly to you so I'm sorry to write in here (you can delete it after reading it) but I think you should know about this project we worked using your library.

All the issues I had was during this project so I though I might just drop you a line to, first thank you for the library you wrote which made it possible and secondly send you the link of the games we manage to pull using it.

Basically we came up with this idea for Movember (if you dont know what it means check the website www.movember.com), using face recognition to play games with your "Mo". Well, after a refined search across the web the best library we could find was yours. Even thou we are in early stages with face recognition, yours was the one that stand out so we than decided to use it.

We created 4 HTML5 games for Movember on http://www.movembergames.com/.

The developers of the games are

Lame-mo
http://www.movembergames.com/lame-mo/
Rafael Cardoso

Fumanchu
http://www.movembergames.com/fumanchu/
Christian Schlosrich

Handlebar
http://www.movembergames.com/handlebar/
Rafael Cardoso

Wizard (Pinball)
http://www.movembergames.com/wizard/
Roger Chapman

I want to say THANK YOU, for spending time on such great library! :)

Update NPM

Hi,

Please could you publish the latest version to NPM? It looks like it's still on v0.0.1?

https://www.npmjs.com/package/headtrackr

Thanks!

Associate name with faces

Is possible to associate names to different users to do some kind of face authentication?

Example:
http://mhuan.name/wp-content/uploads/2010/11/1.jpg

I think that in 'facetrackingEvent' could have some attribute to identify each face

Suggestion: add setOffAxisProjection method to THREE.PerspectiveCamera

I imagine THREE.PerspectiveCamera could have a function like
setOffAxisProjection(x, y, z, screenWidthPixels, screenHeightPixels)

x, y, and z refer to the viewer's position in centimeters relative to the center of the screen

From controllers.js it looks like all you need are wh, and ww. These are pixel measurements, right? That's my guess based off THREE.PerspectiveCamera.setViewOffset

The code that currently refers to fixedPosition could just inspect camera.position. If this seems like a bad idea, then fixedPosition could also be passed as an argument to setOffAxisProjection.

For a related issue, please take a look at this StackOverflow question.

no camera found

Following your instructions in "Usage" i get a "no camera found" on Chrome 33.0.1750.117 ......and looking at headrackr.js i don't see anything wrong.

it passes "if (navigator.getUserMedia) {"

and then goes to the error callback of "navigator.getUserMedia"

any idea why this could happen?

Webworker

Hei Audun,

I was thinking, could a webworker version of the (sweet!) headtracker improve performance?

j.

Examples all missing

All 404:
https://auduno.github.com/headtrackr/examples/facetracking.html
https://auduno.github.com/headtrackr/examples/sprites_canvas.html
https://www.shinydemos.com/facekat
https://auduno.github.com/headtrackr/examples/targets.html

face recognition

Hi, i know this is not an issue. i want to do face recognition with js. wonder if is possible to port opencv with emscripten. any thoughts? do you suggest me any js lib? or procedure to port opencv facerecognizer to js? How have you done to use face detection with js? Thanks you very much

facetrackingEvent not updating event.angle

From the docs/reference it seems as angle should update when rotating my head, just as the x and y values do when I move. But no matter how I move, the angle is always at the same value:

My code looks like this:

document.addEventListener('facetrackingEvent', 
    function (event) {
        console.info(event.angle);
    } 
);

Can anybody tell me wether this is correct behavior (and I'm misunderstanding something) or if this is a bug?

Thanks!

Add possibility to track face in images, canvas

It needs to be possible to track a face in images and canvas-elements as well as video elements.

Support generic pattern detection

Hello.

How about providing a way to detect generic patterns (generated from static images, maybe) as opposed to just faces?

For example, say I want to detect a letter or a hand.

Any plans for such a scenario?

Accuracy

Hi mate,

sorry to bother you again with all these questions.

I notice that you are tracking the face but the neck is also being tracked. Is this being tracking by the contrast or something like that?

If I cover my neck and have only a small that gets from my schin to the top of my head what I want to do works just about right.

Any directions?

THanks

IE Flash Fallback

Hello,
I am using https://github.com/addyosmani/getUserMedia.js to stream webcam in IE. But it is only streaming in flash when used in IE. And when i try to pass flash object to the headtrackr it is not working?
I couldn't find a workaround to copy the stream of the Flash to video element. Anybody knows how to do it?

Inconsistent event names in reference

Hey @auduno

thank you for this great library. I'm using it on one of my projects.

I noticed some inconsistencies in the reference: in the second paragraph, it says facetrackerEvent and headtrackerEvent are triggered, when they're actually called facetrackingEvent and headtrackingEvent. It took me a while to figure out that my script was listening to the wrong events.

Would you mind updating the event names reference?

(I'd send a pull request, but for some reason github won't let me...)

Thank You!

Could this be used for a people counter?

Apologies if this is not a very 'githuby' question but I dont know where else to ask...

I am just wondering if this could be used to count the number of people walking past the webcam?

Any tips or ideas would be super appreciated

How do I horizontally flip the video?

How do i mirror image/horizontally flip webcam???

reset?

Sometimes it loses the face and can't get it back without reloading the page. Is there a way to manually reset the tracking portion?

htracker.stop() does not work from an eventlistener

when calling stop() on a headtracker instance from an eventlistener, an InvalidStateError is raised in Chrome:

Uncaught InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable.

Mouth tracker

Hi mate,

Sorry to bother you with all these questions, but have you ever manage to do any testing to do a mouth tracking?

I'm no nerd or great with math so I was wondering if you could help me out.

thanks

Different type of Haarcascade

Hi everyone,

I would like to know if there is a way to build the same haarcascade object (headtrackr.js line 841) from other haarcascade files (like opencv xml files for instance). It would be great to be able to detect other part of the body/head without writing cascade object by hand. Generate them would be wonderful.

Thanks in advance,

does not work in firefox

I believe the problem is in this line, should use video.srcObject = stream; instead.

How to run this project

How to run this project on your web browser after installing all the dependences.
I have installed all the dependencies, but don't know from where to start the project

Dark/Bright detection

Hey mate.

Is there any way to detect if it's too dark? or too bright?!

Lets say someone is using the webcam but the lights are off, is there any way to tell there isnt too much light to use the facetrackr?

thanks

Uncaught TypeError: Failed to execute 'setAttribute' on 'Element'

I get the following error when using headtrackr:
Uncaught TypeError: Failed to execute 'setAttribute' on 'Element': 2 arguments required, but only 1 present.

on line 29:
videoInput.setAttribute('autoplay');

Option to not output to canvas

Would be great to add the option of not having to render the face tracking to the canvas element, in case you only need the data from the events.

for example

htracker.init(videoInput, false);