Visual Search Moves Beyond Experimentation and Into Prime Time

This post is the latest in our “Visualizing Local” series. It’s our editorial focus for the month of May, including topics like artificial intelligence and marketing automation. See the rest of the series here. 

After years of experimentation and broad discussions about how visual search would someday take hold, it’s clear that the future has arrived. Visual search has moved into the mainstream, and companies like Pinterest, Instagram, and even Google are paving the way for consumers to engage more deeply with the products they find online.

Having taken notice of the shifting landscape in search marketing, the largest technology players are responding with an array of sophisticated visual search tools. Google, Amazon, Alibaba, eBay, and Snap have each revealed their own visual search tools in the past year, building on the improvements in computer-vision technology to make the dream of mainstream visual search a reality for both consumers and brands.

In a 2018 survey from ViSenze, 62.2% of “younger millennials” (between the ages of 18 and 20) cited visual search as a new technology they would be comfortable with as part of the digital shopping experience. An almost identical 61.7% of “older millennials” (between the ages of 21 and 34) said the same thing.

In a separate survey, this one by the Intent Lab, 59% of shoppers said they believe “visual information is more important than textual information.” Within specific product categories, visual information ranked even more highly. For example, roughly 85% of shoppers said they place more trust in images than words when they’re shopping for apparel and home furnishings.

That sort of broad consumer adoption—and consumers’ willingness to accept visual search technology, at least in part because of the early investments made by companies like Google and Pinterest—has pushed more technology firms to jump on board.

At the social commerce firm Curalate, CEO Apu Gupta sees visual search increasingly making its way into mobile shopping environments, enabling people to find similar products to the ones they may encounter in the real world.

“Shoppable images are becoming the new storefront and a core part of our content experiences on social,” Gupta says. “As consumers increasingly engage with shoppable content in channels like social, where they spend a lot of time, they’ll increasingly expect that they can arrive at any image regardless of where it is, tap on it, and see what’s inside of it.”

As visual search moves into the mainstream, questions are intensifying over what impact the medium will have on SEO and traditional search metrics.

Visual Search’s Influence on SEO

At Constructor.io, a firm specializing in search science and artificial intelligence, CEO and co-founder Eli Finkelshteyn has had an up-close look at the development of visual search and the push from consumers for more of this technology. Pinterest, for example, now has more than 600 million visual searches per month, with a number of discovery tools, like Shop the Look and Pinterest Lens, that are designed to help users engage with content visually.

Because the visual search industry is developing so rapidly and the field itself is so new, a lot of the SEO implications experts say visual search portends are just best guesses, Finkelshteyn says. Nonetheless, he believes that structured text will still be important going forward, simply because of the way some visual search engines have been designed.

“For some visual search engines, image features are turned into structured text, so structured text on images may still be important,” he says. “Another thing likely to be important is ensuring an image you want matched on is ‘most’ similar to the way users would search. For example, making sure a picture of a landscape you want to do SEO on is taken at the same time of day as when most visual searches would occur.”

Pinterest has been noted as having increased its use of visual symbols inside search results, thereby decreasing its reliance on text over time.

Google, for its part, is also taking visual search seriously as a potential game changer for traditional SEO. As Damian Rollison wrote in Street Fight earlier this month, Google introduced a number of AR-based features with implications for visual search at the 2019 I/O developer’s conference. For example, the company is using Google Lens to create “smart” restaurant menus that link to photos and review content from Google My Business.

The goal for Google appears to be bringing visual information directly into search and letting consumers use their smartphone cameras to turn their phones into robust search tools.

Will Social Dominate Visual Search?

Who will ultimately “win” visual search? The answer depends on whom you ask and what they believe “winning” looks like in the visual search space.

Street Fight Lead Analyst Mike Boland believes the smart money is on Google given the decades of work the company has already put into its vision technology.

“The thing that will ‘win’ visual search isn’t the front-end experience but the back-end object recognition, computer vision, and machine learning,” Boland says. “Google, with its knowledge graph [and] object recognition, blows everyone else away.”

Pinterest’s visual search strategy has been evolving over time as well. Consumers have already demonstrated their comfort with visual search as they use Pinterest tools like Shop the Look to find “pins” with similar products. Pinterest also offers Lens, which is a real-time visual search tool that’s embedded in the camera in the company’s mobile app. Between February 2017 and February 2018, Pinterest saw searches on Lens increase 140%.

Constructor.io’s Finkelshteyn sees platforms like Instagram trying to keep pace with Amazon, which many view as a leader in the space.

“In a world where more than half of users start their product search at Amazon and 74% go to Amazon when they’re ready to buy something, Instagram’s visual search is a welcome relief for everyone else,” Finkelshteyn says.

Instagram is one of a number of social platforms to use pictures to capture sales for brand marketers. The company announced Checkout on Instagram earlier this year in an effort to find a way for users to purchase products without leaving the Instagram app.

“Instagram knew its strength is images, and allowing users to use those images to search for and buy products without ever leaving the site is both a boon for it and for e-commerce that happens outside of Amazon,” Finkelshteyn says.

A Path Forward

Having moved out of the experimentation phase and into mainstream adoption, the future of visual search is set to rely heavily on technical innovations that improve accuracy and speed. At computer vision companies like GumGum, visual search is being fast-tracked and algorithms are being refined.

At present, Finkelshteyn says a lot of the problems for visual search are the same as those impacting the greater adoption of voice search—the best UI and mediums have probably not yet been invented.

“It’s not that loads of people are trying visual search and not getting the results they want, but that not enough people are trying it yet because they don’t have compelling reasons to do so,” he says. “In the next five years, we’ll see a ton of innovation on new mediums in visual search. Companies like Instagram and Ikea are already trying interesting new uses of it, and those much more than algorithmic improvements will have the biggest impact on the field.”

Stephanie Miles is a senior editor at Street Fight.