Introduction to creative coding and computer vision using modern web based technologies. Computer vision is with us since many years. Reading and tracking light, pixels, blobs, edges, colors are common techniques in interactive environments. However, in recent years, the field is just shifting to next level states with successful learning systems, like machine learning and deep neural networks. Semantic content became available to deal with, thanks for the extended databases and training algorithms out there. Our machines now understand drawings, objects, human poses, gestures, and many other semantically meaningful components, that are related to vision. This workshop covers some basic methods on how to use these advances, also, it is intended to find novel ways of cognitive interaction modalities based on these tools.
We are using the free and open source p5js library to create shapes and forms on the canvas, in Javascript language. The simplest way to get started is to use the excellent online p5 editor, however if you plan to bring your pieces offline and continue working on them in a more elaborative way, it is useful to clone this repository and use the open source, cross platform editor VSCodium with your projects.
001 - uses simple webcam input + p5js
002 - 006 using ml5js and neural networks to learn to find custom features in the image
007 - using mindAR (on top of A-Frame) as the computer vision engine and displays content in mixed reality
001 - color tracking
002 - object detection
003 - posenet
004 - hand pose
005 - feature extraction
006 - pixel regression
007 - augmented reality + image tracking - open this on your phone, then show this image to your device. Multiple videos example
Tarot Card Divination (Boszorkány Project). Works with this this marker image shown to the device
Skill Cards Works with this this marker image shown to the device
Garbage Identifier Works with this this marker image shown to the device
Augmented Posters. Works with this this marker image shown to the device
Community Garden. Works with this this marker image shown to the device
Classroom Schedule. Works with this this marker image shown to the device
Human Pose and motion: an extensive overview of the topic in media history, by Daito Manabe
Open Source Machine learning Projects: what people are doing with ML in 2018
History of Machine Learning: general overview starting from 1642
Artists and Machine Intelligence: medium publications from different artists working in the field