jodeleeuw/219-2021-eyetracking-analysis

Hi there 👋

I am a cognitive scientist 🥼 at Vassar College working on developing research software to make behavioral experiments easier, more accessible, and cheaper. My current interests are focused on how we can use deep learning to improve behavioral experiments, scientific workflows, and classroom experiences. I also occassionally study learning in classroom and laboratory contexts and build video games 🎮 that incorporate cognitive science experiments into the gameplay. I teach classes on cognitive science, cognitive modeling, deep learning, research methods, and robotics 🤖.

I created and maintain jsPsych, an open-source JavaScript tool for creating behavioral experiments that run in browsers. I recently built DataPipe, a tool for connecting web experiments to the Open Science Framework so that data can be sent directly to the OSF when it is collected.

I'm currently working on building a deep-learning based eye tracking system to enable high(er)-precision eye tracking 👀 in online experiments. I'm also thinking about how to facilitate more collaborative development of experiments through open source ecosystems and creating LLM-based tools for structured conversations in classroom environments.

A complete list of my published work is available on Google Scholar.

219-2021-eyetracking-analysis's Issues

Compare replicability to calibration success

Group D write-up: Comparison to in-lab data

Group D write-up: Strength of results in comparison to behavioral data

calibration data questions

@raryskin @jodeleeuw @eforbes24

In looking at group c, I saw that calibration analyses focus on the "boost calibration" phase. Group e doesn't have that phase at all. A very small number of subjects went through "recalibration", but I'm not sure what that means. Was that immediately following the initial calibration if initial calibration wasn't good enough?

For now, I'm assuming that is the case, so it makes sense to use the second calibration score when there is one.

Group A write-up: Comparison to calibration data

Group B: Strength of results compared to behavioral data

Group D write-up: Basic methods and results

Improvements to webgazer.js

You'll find a 'FUBAR' in "General Methods". You made some improvements to webgazer.js to improve timing, and I think we should mention those here. Possibly referencing the issues that others have reported.

Group B: Comparison to in-lab data??

Do we have any?

Title & author list

Can we please decide a title and authorship list in the next week or two. I don't care what it is, but I do want to be able to cite this in my tenure materials when I submit at the end of the month!

Post Altmann & Kamide data for JKH to run power analysis

Group E: Comparison to calibration data

Group D write-up: Comparison to calibration data

Group A write-up: Strength of results in comparison to behavioral data

Group C write-up: Comparison to calibration data

Group E: Basic methods and results

New data collection?

Will we replicate any of the studies in the lab or online and, if so, which ones?

Will we run the posner task?

Group A write-up: Basic methods and results

Initial write-up

Group C write-up: Basic methods and results

Sample rate question

@jodeleeuw -- is sample rate for data collection determined by jsPsych? WebGazer? The camera? (I mean, there's the sampling rate of the camera, but I mean who decides how often data stamps occur?)

Group B: Comparison to calibration data

Group E: Comparison to in-lab data??

Do we have any??

Selection of studies

I started some text on how the five studies were selected. Right now, it needs a description of the five studies. It might be nice to include info on number of citations (justifying why these studies and not others).

Group E: Where is the norming data?

@jodeleeuw the methods mention norming data, but I don't seem to have them?

How was number of subjects determined

From "General Methods":

In order to have adequate statistical power and precision, we aimed for XXXX participants per experiment.

Group A write-up: Comparison to in-lab data

Do we have any??

Group B: Basic methods and results

Is there a long lag time?

Was talking with Jesse Snedeker yesterday. Her lab has been using Webgazer and finds the time stamps are delayed by about 300 ms. Apparently, they followed this up by actually running Tobii data through it (??? not sure what she meant) and found the lag was still there, so she thinks it's about the way analysis is done.

Do we see anything like that? Would we know?

Why were these subjects excluded?

Group-E:

all.data <- all.data %>% filter(subject_id!="603ea6a51c38def3e4140ba6" & subject_id!="5ce1372c55fe0d001869c8d8" & subject_id!="5f47d860b56e68220cb86be1" & subject_id!="607eaf87ac02229ef4fa8f4a" & subject_id!="5d30857eb2f0ad00163470f3" & subject_id!="5e272684f205790453c2c4b8" & subject_id!="5f5b5881b17fa72756b8ff2a" & subject_id!="5fae5291ec402509e2885604")

Group C write-up: Strength of results compared to behavioral data

Group E: Strength of results compared to behavioral data

Group C write-up: Comparison to in-lab data??

Do we have in-lab data for this?