Evaluate on behavioural data

[lukavdplas]

Evaluate on behavioural data

[lukavdplas]

Something to keep in mind:

The frequency table gives frequencies per genre/category of programming, so we can evaluate different subsets. T-scan is currently based on SUBTLEX-NL which uses film subtitles. We may expect the film categories in the NPO data to be the closest match, though this subset is relatively small.

In addition, the whole purpose of using the NPO data is to include more recent vocabulary in t-scan. For various reasons, films have a significant delay between a word entering the public discourse and it appearing in films for the first time - several years at least (though films also have a tendency to avoid vocabulary with an expiration date, which makes them biased against recent vocabulary in general). This delay is increased since we are using films that appeared on TV, which means they are often older.

Television is generally faster - pre-recorded programming like documentaries can show a delay of several months, while live programming like news and talkshows can have no delay at all (and can, in fact, be the agent introducing the word to public discourse).

This kind of makes films the least useful option for us. A common word like "app" occurs thousands of times across the entire dataset, but only twice across all films.

The behavioural evaluation is based on an older experiment though, which means that it is biased towards the more "timeless" vocabulary of films. The frequency of recent words like "app" would only throw off predictions.