Foursquare’s New Audio Assistant is a Peek into the Future of Local Tech

In Spike Jonze’s 2013 science-fiction film Her, the protagonist Theodore walks around a fair at night with his eyes closed. This isn’t a pandemic-inflected world; the fair is crowded, and yet Theodore bumps into no one as he spins, walks, and sneezes without sight. Guiding him is Samantha, a virtual assistant who understands both his surroundings and desires — so well, in fact, that the scene culminates with Theodore standing before a vendor, saying “I’d like a slice of cheese, please.” Theodore opens his eyes to realize he has just ordered something to eat. “I figured you were hungry,” Samantha says.

This is perhaps the fully realized, future iteration of a new technology debuted this week by location intelligence firm Foursquare. Dubbed Marsbot for Airpods, Foursquare’s virtual assistant will whisper insights to users about their surroundings, unprompted, as they move throughout the world. Marsbot’s insights could be a recommendation for a local coffee shop or a fun fact about a landmark. They could also take the form of sound effects, and the app will let users know when they happen to cross someone else using Marsbot.

The app is currently available in Apple’s App Store, and it will eventually be available on Android devices. It will work for all kinds of headphones, not just AirPods, despite the name.

The promise of Marsbot is similar to that of AR-enhanced glasses: Something we carry on our person is imbued with knowledge about our surroundings, seamlessly boosting our awareness. But the existing widespread consumer adoption of headphones capable of connecting to smartphones is precisely the difference between Marsbot and smart glasses, which have been notoriously slow to catch on, said Foursquare co-founder and executive chairman Dennis Crowley.

“Do you have magic AR glasses? No,” Crowley said. “So let’s just build for the things that people have.” Calling Marsbot an intermediate step between the old days of Microsoft’s Clippy and Her‘s futuristic Samantha, Crowley said the technology is just ambitious enough to be immediately useful and usable while also opening “the door so you can glimpse the future.”

For brick-and-mortar businesses and the technology providers that help them connect with customers, the marketing possibilities are tantalizing.

“This is an inspired idea,” said Greg Sterling, VP of market insights at location marketing firm Uberall. “Foursquare could and should also turn this into a multi-platform tool, so if you wanted to mute audio recommendations but still wanted to receive them, you could get them as texts.”

Personalized, on-the-go texting recommendations were the stuff of Marsbot’s first life. Years ago, the project did just that — send users recommendations via text based on location data — and Crowley said Foursquare developers will likely turn Marsbot’s current iteration into the “multi-platform tool” Sterling envisions if the audio assistant takes off. Ads are also on the table, though Crowley said Foursquare is not actively working to monetize the interface at present.

While the audio assistant could be a boon to local advertisers, a Her-like future in which Marsbot is more knowledgeable about both its surroundings and users’ preferences also summons fears of a dystopian future. It is not hard to see the thin line vanish between an assistant that is helpful and one that is controlling. Frameworks like those developed by Shoshana Zuboff, the theorist of surveillance capitalism, train us to see those two types of relationships between “smart” technology and users — “helpful” in the parlance of firms like Foursquare and Google, “controlling” in that of activist academics — as one and the same.

On a more commercial note, another area of concern with advertising on Marsbot for AirPods would be a disruptive or overwhelming user experience. Sterling said one could imagine the technology annoying users, either because it pings them with too many notifications or because it provides them information they do not need for a pre-planned trip.

Key to addressing both these concerns will be feedback from users as well as their own control over the user experience, Crowley said. If the device is too “chatty,” Foursquare expects to receive that feedback and adjust the service accordingly.

As for privacy concerns, recommendations will be based on the same user and location data that already powers Foursquare’s products. As an example, Crowley cited the feature on consumer-facing app Swarm that tells users if they have forgotten to check in after visiting a restaurant. Marsbot also lets users know that it’s on so that it does not suck up background location data undetected. What’s more, if users are uncomfortable with the data Foursquare collects about them or the personalized services Marsbot may one day provide, the company will allow them to delete data and disengage from the app.

“It’s a privilege for us to have your data,” Crowley said.