Tom Bastek | April 09, 2015 11:28 AM ET
Why Most Travel Surveys are Unreliable
This week, Yahoo Travel released the results of a survey was which told of the most overrated cities in the country. Las Vegas was number one followed by New York and Miami. The article goes on to talk about why each city is such a letdown.
Before I talk about surveys in general and why most of them are really just headline attention grabbers, I am going to defend Las Vegas here. The line from the writer of the article about the survey is and I quote, “But let’s be honest: Any city where you can lose your life savings in 30 minutes can’t be that great.”
I understand not everyone sees value in Las Vegas and that is fine, but this is one of the most irresponsible statements ever made about the city and here’s why:
1. You can lose your life savings, if you are ignorant enough to spend it all, in any of the more than 1,700 other casino in the United States or any of the 4,490 casinos located on any of the other five inhabited continents (although I am sure it is just a matter of time before Antarctica gets a poker machine).
2. There are 1,000 other things to go to Las Vegas for besides gambling. Even the suits running the resorts are smart enough to have diversified because they know that gambling is not the main reason tourists flock there anymore.
The lines in the article above are not the real problem here; it is the lack of data upon which they base their survey results.
Sample Size Me
We are in a world of technology; a world that provides us with so much data that we will have a hard time sifting through it all on a real-time basis. Surveys in general, and especially when they deal with travel, are extremely hard to trust with such a small sample size.
The current population of the United States is a little over 318 million people. The 2,000 people that were surveyed make up just .006 percent of the public. Now, I am sure that not every single person in the country traveled domestically last year, but I am sure at least half of the American population did. What I am sure of is that on the average a typical American has taken fewer vacation days over the last five years, and traveled to fewer places. So if the average person went to, even on the high side, five different cities each year for vacation, they could really speak to about their personal experience in 25 different cities over the last five years.
So how can you poll 2,000 people who, we have to assume, have actually been to the city that they are declaring overrated, and push that out as legitimate data? This is the primary problem with surveys of this nature: insignificant sample size.
Most people have an expectation when they travel, go out to eat, even when they go to the grocery store. And that expectation is different for everyone. One of the reasons that review websites are so misleading is that to really use their data properly, you have to get into the nitty-gritty and find out why people did and did not like about said product/service/experience.
Anyone can say they had a four-star experience at a restaurant but we have no idea why/how/when. The same would hold true for the survey that told of, “Cities with the best shopping.” The subjectivity of these surveys is so diverse, that no one can hold this data as true.
See Related: 7 Things To Know Before Reading A Review Online
Poll vs. Stats
The much more credible data comes from past statistics, because they contain verifiable facts. Do you want to know how popular Las Vegas is? Look at the number of people that travel there every year. According to the Las Vegas Visitor Authority, over 41 million people came to Las Vegas last year. This number is an increase over the previous, and Las Vegas has actually experienced growth every year for the last five years.
This to me is not the sign of an overrated city. To ask 2,000 people to vote on the most overrated city in America is tough. A much better survey would be to list the places you have visited in the last five years and then rate them on a scale of 1-10 with 10 being the most overrated. Then you could continue to build your pool until you had enough data to fill it.
Even though the typical consumer today is smarter than any of their predecessors, their attention spans are so short that they live and die by the titles of articles and the bullet points within. They don’t have the time to read a two-paragraph review to find out why the restaurant was bad, they just notice that it only got two stars so they better not go.
They don’t know why a city has great shopping, they just know that there were three surveys that they found online that said, “Atlanta had the best” and so that is where they are going to go.
It makes sense that marketing departments for corporations are always going to put their company’s face in front of any stats or polls that they put out. In order to fix the system, the independent companies have to be the mediators; the referees. It is up to them to seek out real sample sizes, deep understanding and factual statistics to back up their headlines. Because as heavily read as Yahoo is, if I was Las Vegas, New York or Miami, I would be very concerned about the effect of a poorly conducted poll on my tourism numbers for this year.
Get Travel Deals and Travel News
Latest Travel News
Airlines & Airports
Hotel & Resort
Destination & Tourism
Airlines & Airports
Airlines & Airports