The project is based on the Reactivision software. Which takes care of recognising a specific set of amoeba looking symbols and reporting their position.

The underlying tech is fairly simple, though getting it right (or at least close) turned out to be a pretty big job. The surface itself is a perspex sheet with a sheet of filter applied, so it can hold a projected image. Underneath the screen is a camera, to recognise the objects placed on it, and a projector, to display the interface. There is also an infrared light source and the camera is filtered to allow in IR only, so it isn’t confused by the light from the projector.

The interface is certainly only a fraction of what is possible with this system. The majority of my time went into the physical setup and I wanted to get something done, so this is a quick hack in Processing.