Guardian study finds inferior items appear highly praised, making ratings worthless
After the Guardian got in touch with Amazon it separated some of the mismatched reviews that had been brought to its attention. Guardian study finds inferior items appear highly praised, making ratings worthless
Badly translated versions of classic books and critically panned remakes of Hollywood films appear to have glowing endorsements on Amazon thanks to the website’s policy of bundling together reviews of different products.
Analysis by the Guardian shows products that have actually been given one-star ratings appear alongside rave reviews of better quality items, making it impossible for consumers to judge the true value of what they are about to buy.
The Guardian found numerous examples of “bundled” reviews that make poor products look highly rated – rendering the star rating effectively meaningless.
The research found:
• Badly translated or updated Kindle versions of Emma by Jane Austen and Charles Dickens’ Great Expectations, which include references to “moms”, “guys” and “buddies”, but appear to have 4.5-star ratings from hundreds of reviewers.
• A 2017 TV version of Dirty Dancing that shares the 4.5-star reviews of the original film, despite being described by Hollywood Reporter as a “bloated” remake “that nobody asked for and nobody is likely to truly enjoy”.
Reviews for Wuthering Heights appearing under listings for Jane Eyre, and vice versa.
•Complaints from consumers who said they had been misled when buying books from a variety of authors – from JK Rowling to Shakespeare.
• Star ratings being combined for different products in other departments, from electronics to gardening equipment.
The problems with some reviews seem to go back years, with complaints from readers pointing out they were appearing under the wrong works and editions since at least 2014.
The combinations of formats and editions make it impossible for readers to pick between multiple versions of the same products, and allow those selling badly put together editions to piggyback on good reviews.
Anyone glancing at the reviews for a Kindle version of Emma retailing at £4.36 might believe it is worth buying, but a look at the opening pages reveals a poor translation of the original.
Emma’s mother has become her mom, and her love interest, Mr Knightley, is “a sensible guy” who uses the word buddy instead of friend.
A passage that is supposed to say “poor Miss Taylor” will be missed, instead reads: “She is surely very sorry to lose terrible Miss Taylor, and I am positive she can leave out her more than she thinks for.”
The same is true of a Kindle version of the Dickens classic, selling for 91p. Far from meeting the great expectations the 4.5-star review might imply, a glance inside suggests the rave reviews do not apply to this version.
A review from a reader, which appears to be about this edition, gives it just one star and describes it as terrible.
“Each page has a dozen errors. It reads as if it has been translated from a foreign language. ‘Dog’ in the original is ‘canine’ in this version; ‘file’ in the original has become ‘document’; ‘tremendous’ has become ‘maximum incredible’; ‘man’ has become ‘guy’.
“That is just a short summary of the errors in the first two pages. The whole thing is unreadable and a waste of money.”
As well as a mixture of formats, customers are being misled by ratings based on completely different stories or productions.
Reviews for Jane Austen’s entire oeuvre seem to have been combined, and bundled with related books by other authors, which means anyone looking to see how a story is rated for itself, is likely to be misled.
An updated version of Austen’s Emma, written by the author Alexander McCall Smith, appears to have a 4.5-star rating on Amazon, but on close inspection is linked to the same reviews as all of Austen’s work.
The reviews that mention his name give an average of three-and-a-half stars instead, while on the review site Goodreads the book is rated three out of five.
A 2017 TV version of Dirty Dancing, which was critically panned and on film review websites gets just one star on Rotten Tomatoes and 1.5 on the Internet Movie Database, looks like it has 4.5 stars from more than 1,000 Amazon customers.
The recent Kenneth Branagh remake of Murder on the Orient Express is grouped with a more acclaimed 1974 version starring Albert Finney, while a 2017 remake of the film Flatliners has its reviews combined with the original.
By mixing the reviews, it gives the impression that the modern film, which was described by one critic as “unrevivably dead”, has been highly rated by Amazon customers who were commenting on the 1990 movie.
Shakespeare’s plays suffer the same fate as Austen’s work, so anyone wondering whether to buy or not to buy a paperback copy of Hamlet will find it shares reviews with Kindle versions of Othello and Macbeth that turned out to be in German.
Most of the reviews do not mention which play or edition is being appraised, making them meaningless to consumers attempting to pick between different options.
Every Harry Potter book in JK Rowling’s series shares reviews too.
It is not just the works related to one novelist that are being combined.
Until the Guardian got in touch with Amazon, reviews for Emily Bronte’s Wuthering Heights and Charlotte Bronte’s Jane Eyre were thrown together, with the result that anyone unfamiliar with the novels would struggle to know which is being recommended, or not.
And ratings for Henry James’s The Turn of the Screw are combined with those for Mary Shelley’s Frankenstein, so it is not clear which is being given a monstering in many of the one-star reviews.
Consumers have left comments suggesting they have been misled by the system.Under a collection of reviews for various versions of Lewis Carroll’s Alice’s Adventures in Wonderland, one customer writes of disappointment on receiving the book they have ordered: “Despite the first few reviews which lead [sic] me to believe this book had the original illustrations; this book has no illustrations.”
It is a similar story over at the Harry Potter listings, where a recent review says: “After reading feedback and customer uploaded pictures I thought this set would contain books with pictures inside.”
Meanwhile a reviewer of one of Shakespeare’s plays bought on Kindle says: “This is a scam. It comes up at the top of the search with many positive reviews but it’s unreadable. I am still trying to get my money back.”
It is possible to separate reviews by format, if you drill down into each rating level, but the star system still shows the average for all reviews. And it is not possible to separate reviews by publisher.
Efforts to clean up reviews in other parts of the site mean it is now clearer which products comments are about, but there are still cases where reviews for different items and their star ratings are being combined for everything from doorbells, to phone cases and lawnmowers.
The consumer group Which? said its surveys showed 97% of its members used reviews when buying a product, and 30% had been disappointed by products they had bought after seeing good reviews.
Natalie Hitchins, the head of home products and services at Which?, said: “If online retailers are incorrectly grouping customer reviews for different products together, there is a real risk that their customers will be misled.”
She added: “We need to know we can rely on the information that retailers provide about the products they sell. Online retailers must up their game and ensure reviews are presented in a clear and accurate way.”