Uber unveiled this week that it’s been running a pilot program since November that tracks driver behavior via the driver’s phone in order to determine the validity of passenger reviews. Using GPS and accelerometers in driver phones, Uber thinks it can double check consumer reviews that claim a driver was driving erratically, speeding up too quickly, or stopping short at too many lights. Based on its analysis of the data, the company may decide to remove a review it deems inaccurate. Conversely, Uber may send disciplinary messages to drivers whose phone data backs up consumer claims of unsafe driving.
Just as email gave rise to spam and online journalism gave rise to clickbait, consumer feedback, one of the more valuable properties in local media, birthed the phenomenon of the fake review. Yelp, in many ways the gold standard in local reviews, built a recommendation engine to detect traits that signal a review might be fraudulent. Yelp flags users who have only published one review, reviews that are too strongly positive or negative, and other factors, and filters out reviews that do not meet the recommendation engine’s criteria.
But numerous complaints over the years from businesses and consumers attest to the imperfect accuracy of Yelp’s filter. After all, someone who has only posted one review is not necessarily faking it. As with most algorithms, Yelp’s recommendation engine relies on likelihoods based on a statistical model, rather than measurable certainties.
Amazon has built a similar reputation on the strength of its consumer product reviews, and the company has a fact-based model of sorts in its Amazon Verified Purchase feature, which places a badge next to reviews when Amazon can associate the reviewer with an actual purchase of the product under review. Like eBay buyer and seller reviews, all of which are tied to a transaction, Amazon’s Verified Purchase feature helps ensure that the reviewer is qualified to speak about a product. Verified Purchase badges, along with “helpful” votes from other Amazon users, top reviewer status, and other signals, determine which reviews are featured on a product page.
Still, the prevalence of fake Amazon reviews is enough of a problem to bring about services like Fakespot, a website that analyzes the review content on any Amazon product page and calculates the quality of those reviews, giving low quality ratings to reviewers who only write very positive reviews or who have written several reviews about products from the same company.
Review Skeptic, a tool built by researchers at Cornell University, scrutinizes hotel reviews in a similar fashion. The researchers claim they have achieved 90% accuracy in detecting fake hotel reviews by teaching their software to detect signals in the review content itself, gleaned through analysis of hundreds of real and fake reviews from TripAdvisor.
Again, these are algorithmic solutions to a problem that could potentially be solved with empirical data. It’s intriguing to think of how this problem could be addressed in regard to local businesses. Uber’s data-driven verification model does not attempt to assess the accuracy of every element in a review; only those elements that relate to the movement patterns of the vehicle, as implied by the movement patterns of the driver’s phone, are subject to verification. But you could argue that greater the number of verifiable facts in a review, the more accurate the overall review is likely to be.
For Uber, review verification is probably just one of many applications for gathering data from phones. The same holds true for more stationary brick and mortar businesses, where transactional data, beacon and wifi signals, GPS, and similar location-based data points can be correlated for a range of uses, from hyperlocal advertising to in-store assisted search. The same types of data, as they become more widely available to businesses, could be used to verify that a reviewer actually visited a store, went to the department where a given product is sold, and purchased that product before leaving a review. Similarly, service oriented businesses could scan for signals to verify, for instance, that a patron really visited a given restaurant on a busy Saturday night, and really did wait an hour before anyone took her order.
Of course, the value of reviews is as much subjective as objective. We look for people like us who seem trustworthy and thoughtful to provide evaluative input that helps us decide whether a product or a business is worth our time and money. The key element of any review, the reviewer’s opinion, is unfalsifiable, because it is a subjective response to the facts. But in an atmosphere where fake reviews are all too easy to create, we need tools that help distinguish real opinions from garbage. Moving beyond the limitations of data algorithms, fact-based approaches hold out the promise of grounding review services in observable truth.