NEW WORK ON SITE | DEPTH_EDITOR_DEBUG

This is an excerpt of our submission materials for ENTER5, an Art, Science & Technology Biennial in Prague. All the materials are the product of a collaboration with James George:

In October of 2010, science fiction writer Bruce Sterling gave the keynote speech concluding the Vimeo Festival + Awards in New York. He described his predictions for the future of imaging technology. Regarding how a camera of the future may function, Sterling said:

"It simply absorbs every photon that touches it from any angle. And then in order to take a picture I simply tell the system to calculate what that picture would have looked like from that angle at that moment. I just send it as a computational problem out in to the cloud wirelessly."

One month later, Microsoft released their new video game controller, the XBOX Kinect. Kinect is unique in that it uses a depth sensing camera and computer vision software to sense the position and actions of gamers. A group of developers released an Open Source device driver that allowed programmers to access the Kinectʼs data on a personal computer.

Visualizations of space as seen through Kinectʼs sensors can be computed from any angle using 3D software. When the drivers were made available, online creative software developer communities were flooded with artistic and novel interpretations of this data. The images were often characterized by depictions of people as clouds of dots and wireframes representing human figures moving in time.

In mid-2005, six weeks after the tragic subway and bus bombings in London, New Yorkʼs Metro Transit Authority (MTA) signed a contract with the high-tech defense and military technology giant Lockheed Martin. Lockheed Martin promised the MTA a hightech surveillance system driven by computer vision and artificial intelligence systems. The security system turned out to be vaporware and the contract collapsed under lawsuits. As a result, thousands of security cameras in the New York subway stations sit unused.

It is in this technological atmosphere that we chose to collaborate. We soldered together an inverter and motorcycle batteries to run the laptop and Kinect sensor on the go. We attached a Canon 5D DSLR to the sensor and plugged it in to a laptop. The entire kit went into a backpack.

We spent an evening in the New York Union Square subway capturing high resolution stills and and archiving depth data of pedestrians. We wrote an openFrameworks application to combine the data, allowing us to place fragments of the two dimensional images into three dimensional space, navigate through the resulting environment and render the output.

These prints are selected renderings from this process.

Sources:
Video of Sterlingʼs Talk (relevant portion is from around 00:40:00 onward)
WNYC.org on MTA & Lockheed Martin
Original OpenKinect driver repository
The openFrameworks creative coding developer environment