In 2019 I turned an Airbnb into an Alternate Reality Game (ARG) powered by augmented reality (AR) and hosted by Amazon’s Artificial Intelligence (AI) assistant Alexa. Guests entered the apartment to find a locked cryptex on the table and a diary filled with symbols, codes and esoteric ramblings.
AI, AR, ARG
When unlocked the portraits on the wall spoke, artwork morphed into clues and an antique mirror became a portal to another dimension. The game led to the building’s library and then spilled out into the streets of Mexico and on to an Aztec pyramid, ancient castle and a famous art museum. While regular guests were passively admiring a mural of Diego Rivera, our ARG players were watching his masterpiece come alive. A fetus in Rivera’s Man Controller of the universe grew and shifted to reveal a birthmark, tying Rivera and his wife Frida to an ancient cult. The city itself was filled with mystery and magic.
A Sense of Purpose and Whole Greater the than Sum of its Parts
The players were tourists that surrendered their itinerary to the game. They traveled to locations many tourists visit anyway but with a greater sense of purpose. Regular tourists visit these locations, some might hire a guide to tell them about the history and some of the more adventurous will search the internet for hidden off the beaten track places. But our players experienced each separate location as a part of a greater whole that they pieced together themselves - they were the heroes in a journey with a beginning, middle and end.
How To Make Your Own ARG?: Start with AR
The only app players needed was Zappar, a platform for creating AR and an app to experience them. Most experiences began as simple animations I created with video editing software. Yes, most people associate AR with 3D animation but my preference isn’t to add new animations but rather morph what is there into something new. One of my animations was not created with editing software but by two artists. The first artist, Diego Rivera, though dead, was unwittingly woven into the game’s narrative along with his wife Frida Kahlo.
The second artist was Mayra Hernández RÃos, a film director and artist who worked on animating the paintings of Van Gogh for the film Loving Vincent. She used the same animation process for my augmented reality as Loving Vincent. She would place an enlarged copy of the original down on a table then overlay it with a large pane of glass. Then with oil paint she would paint over the original. While the paint was still wet, she would photograph the result then wipe a small section with a rag and repaint the newly clean area of the glass with a slight change. Photograph again and repeat.
When the player held their phone and activated the Zappar app, the fetus in Diego Rivera’s, Man Controller of the Universe, would grow to reveal that the baby has a birthmark in the shape of the infinity symbol, a clue that solves a riddle in the game and drives the narrative forward.
Alexa Was a Patient and Polite Host and Storyteller
Zappar was the tool for linking puzzle hunt riddles and answers but the storytelling mechanism was solely the responsibility of Alexa. Players weren’t required to have Alexa on their phones and no humans were needed for troubleshooting. Though AI voices can be generated now with platforms like ElevenLabs, I uploaded my actor’s audio files to Voiceflow, a no-code platform for creating voice applications. I was able to integrate this seamlessly with Alexa so that her voice was the base. This way players didn’t know when Alexa was giving game related information or general information. This was done on purpose to blur the line between the game and reality, a key characteristic of ARGs.
Ambient Computing Brings it all Together
ARG creators want to blur the line between games and reality. The evolution toward ambient intelligence makes this easier. Ambient intelligence or ubiquitous computing is when computing fades into the background. It just happens. This website highlights many new tools such as AR and blockchain. To leverage these as storytelling devices we need a host to switch seamlessly between them. This host is an AI powered voice assistant and an important step while learning how to make your own arg.
Compared to a similar experience, like an escape room game, this model is highly scalable. For thoughts on how AI can be used to scale to the power of something like Pokémon Go have a look at my blog on AI and Interactive Fiction.
Watch the intro video for the Mexico City ARG below.