Twitter in augmented reality looks like a living nightmare
Twitter is a rash. It’s a raw mark on your thigh, below your trouser pocket. Scroll and scratch as much as you like, but give it a few minutes and the itch will be back. Still, at least it’s localised…
Not in this experiment from Or Fleisher and Anastasis Germanidis. The pair has come up with a way of spreading Twitter onto everyday objects, bringing floating tweets into the world with the help of augmented reality and machine learning.
The Twitter AR project uses Apple’s ARKit platform, in coordination with the company’s pre-trained machine learning toolkit, Core ML. In a nutshell, Fleisher and Germanidis connected the latter’s image-recognition abilities to the Twitter API as a way surface relevant tweets. Point the iPad at a ballpoint pen, for example, and the software will dig up a tweet that mentions a ballpoint pen.
The result is a world where tweets pop-up around everything from sandals to subway stations. A day-in-the-life clip demonstrating the tech shows a world bombarded with hovering words: breakfast is swamped by ads for snow projectors and links to lifestyle blogs; a busker drowns beneath tweets about pianos; a meeting with friends is interrupted by annotated headphones and inspirational quotes.
“Personally I found small moments of magic when it almost felt like machine learning and augmented reality extended my perceptual senses and brought emotional impacts of objects on to the conscious surface.
“When looking at a fence in the subway station, the classification algorithm predicted I am looking at a prison. Since we disabled the user being able to see what the machine-learning model classifies, it pulled the following tweet: ‘You are the prisoner, the prison and the prison keeper. Only you hold the key to your freedom’ – Ricky Mathieson.”
These small moments aside, the experience looks like a living nightmare. While Twitter forms the basis for the floating text, the experiment is closer in many ways to pop-up advertising than a social network. You can imagine a company like Amazon using a similar mix of machine learning and augmented reality to hawk clothes and household objects.
Still, there is hope that any such corporate efforts to swamp augmented reality with floating ads can be undermined by clever developers – such as the New York-based artist Sebastian ErraZuriz, who “vandalised” a Snapchat AR project.
Source: Prosthetic Knowledge