Ranking Correlations with Other Reputation and Search Metrics Are Not Linear
We’ve conducted a study at SOCi the results of which will be formally released in a few weeks, but I wanted to offer a sneak preview of some of the more interesting findings. We were curious to examine a few possible correlations with how businesses rank in Google local results. For instance, we tend to assume that businesses ranking in the 3-Pack will win greater traffic and conversions than businesses further down in the Local Finder results, but how much of a difference does top placement make? What are some of the other features that correlate with high-ranking businesses, such as average rating, average review count, and presence of featured photos?
Our findings indicate that some metrics and search features correlate strongly with ranking, while others do not. For instance, across all businesses in rank positions 1 through 20, we found that the average star rating hovers very close to 4.2, no matter whether the business is ranking 1st or 20th or somewhere in between. By contrast, some findings do seem to correlate with ranking, but some of these correlations at first seem quite counter-intuitive. For example, the chart below depicts review count by rank position amongst about 8.6 million local search results.
As you can see, review count is far higher at rank positions 1 and 2, drops considerably at rank position 3, then climbs gradually through all subsequent rank positions up to 20, with only minor fluctuations. What could this possibly mean? Why would lower-ranked businesses, beyond rank 2, have more reviews than businesses at higher positions?
The question strikes at the center of what makes ranking analysis so complex. It’s of course self-evident that there is no such thing as “ranking” in the abstract. Ranking means ranking for a particular term or set of terms. The same business might rank at the top for “Italian restaurant,” towards the middle of the pack for “caterer,” and not at all for “basket weaving.” One might assume that ideally, ranking should always correlate with relevance — one of Google’s three local ranking pillars — in such a way that a business ranks best for its most relevant queries, and less well as relevance diminishes.
But examination of local search results themselves tells us something different. We’ve found, across several searches we examined in detail, that Google will sacrifice relevance and proximity in favor of prominence. This is easiest to show with an example.
In the 3-pack result above for “Italian restaurant Staunton VA,” all three of the most prominent results are clearly well-regarded Italian restaurants that are actually located in the town of Staunton. They each have a lot of reviews, and the average rating for the three results is 4.4. Interestingly, review volume bucks the trend in our chart with the third result having more reviews than the second. In this case, the disparity probably results from the fact that proximity to the centroid of the city is being weighted more heavily than prominence.
As a reminder, Google identifies three high-level factors that determine how businesses are ranked: relevance, defined as how well the listing matches what the user is searching for; distance, defined as how far a potential search result is from the searcher or from the place indicated in a search; and prominence, defined as how well-known or well-regarded a business is.
To continue with our example, the Local Finder (what users see when they click “More places” in the 3-Pack) offers additional results, the first three of which replicate the 3-Pack. After these are a group of results from rank positions 4 to 6 that still carry the primary category of Italian restaurant, but begin to feature more restaurants we would probably tend to think of as pizza places rather than Italian restaurants proper. Once we get to rank position 7, we find the primary category of the result has switched to Pizza. After that comes a more distinct break with relevance: the restaurant at rank 8 is classified as Southern.
What we might call the break point — the point where relevance or proximity begins to give way — differs according to many factors. An alternate search for “Italian restaurants Santa Fe NM” finds that Google doesn’t run out of relevant results until page 2 of the Local Finder, beyond the 20th rank position. But overall, the trend seems to be that Google will begin to sacrifice relevance or proximity when it runs out of nearby inventory that is closely matched to search intent, instead offering options that are further away but still relevant, or closer and prominent but less relevant.
Our position 8 example above illustrates this well. Zynodoa Restaurant seems to be a well-liked local example of an upscale Southern restaurant that, although it does not offer Italian cuisine, might be thought of as a reasonable alternative for diners looking for upscale options, as the term “Italian restaurant” generally implies. Indeed, we find that a search for “Southern restaurant Staunton VA” turns up Zynodoa Restaurant as the top-ranked result.
Our observation is that Google appears to think of ranking in terms of zones, where the first zone features the best possible mix of proximity, relevance, and prominence, and the second zone begins to sacrifice either proximity, or relevance, or both, but is less likely to sacrifice prominence. In more human terms, this means that Google wants to show us the best options for a query, and when it runs of inventory, it brings in results that are farther away or that might offer a reasonable alternative. Just where the break point between these zones occurs in actual rankings will differ based on the available inventory for the first zone.
We can now explain why review volume trends upward as rank position declines, as shown in our bar graph. As Google runs out of highly relevant inventory, it pulls in listings that, because of their overall prominence, would rank higher in a different search, just as Zynodoa Restaurant does for Southern restaurants in Staunton. These listings, because they rank highly for their most relevant keywords, attract a high level of traffic, actions, reviews, and more. In slightly out-of-context searches, they appear to outperform higher-ranked businesses, but this is illusory because their performance is mostly based on the terms they rank best for, not on the second-best terms for which they may also appear.
Given this pattern, it would be misleading to attribute any metric-based advantage to rank positions beyond zone 1. Our full study will show that metric-based advantage in terms of increased traffic and actions can be reliably measured up to about rank 10, but not beyond.
