Why Most Travel Surveys are Unreliable

This week, Yahoo Travel released the results of a survey was which told of the most overrated cities in the country. Las Vegas was number one followed by New York and Miami. The article goes on to talk about why each city is such a letdown.

Before I talk about surveys in general and why most of them are really just headline attention grabbers, I am going to defend Las Vegas here. The line from the writer of the article about the survey is and I quote, "But let's be honest: Any city where you can lose your life savings in 30 minutes can't be that great."

I understand not everyone sees value in Las Vegas and that is fine, but this is one of the most irresponsible statements ever made about the city and here's why:

1. You can lose your life savings, if you are ignorant enough to spend it all, in any of the more than 1,700 other casino in the United States or any of the 4,490 casinos located on any of the other five inhabited continents (although I am sure it is just a matter of time before Antarctica gets a poker machine).

2. There are 1,000 other things to go to Las Vegas for besides gambling. Even the suits running the resorts are smart enough to have diversified because they know that gambling is not the main reason tourists flock there anymore.

The lines in the article above are not the real problem here; it is the lack of data upon which they base their survey results.

Sample Size Me

We are in a world of technology; a world that provides us with so much data that we will have a hard time sifting through it all on a real-time basis. Surveys in general, and especially when they deal with travel, are extremely hard to trust with such a small sample size.

The current population of the United States is a little over 318 million people. The 2,000 people that were surveyed make up just .006 percent of the public. Now, I am sure that not every single person in the country traveled domestically last year, but I am sure at least half of the American population did. What I am sure of is that on the average a typical American has taken fewer vacation days over the last five years, and traveled to fewer places. So if the average person went to, even on the high side, five different cities each year for vacation, they could really speak to about their personal experience in 25 different cities over the last five years.

So how can you poll 2,000 people who, we have to assume, have actually been to the city that they are declaring overrated, and push that out as legitimate data? This is the primary problem with surveys of this nature: insignificant sample size.

Semantics

Most people have an expectation when they travel, go out to eat, even when they go to the grocery store. And that expectation is different for everyone. One of the reasons that review websites are so misleading is that to really use their data properly, you have to get into the nitty-gritty and find out why people did and did not like about said product/service/experience.

Anyone can say they had a four-star experience at a restaurant but we have no idea why/how/when. The same would hold true for the survey that told of, "Cities with the best shopping." The subjectivity of these surveys is so diverse, that no one can hold this data as true.

See Related:7 Things To Know Before Reading A Review Online

Poll vs. Stats

The much more credible data comes from past statistics, because they contain verifiable facts. Do you want to know how popular Las Vegas is? Look at the number of people that travel there every year. According to the Las Vegas Visitor Authority, over 41 million people came to Las Vegas last year. This number is an increase over the previous, and Las Vegas has actually experienced growth every year for the last five years.

This to me is not the sign of an overrated city. To ask 2,000 people to vote on the most overrated city in America is tough. A much better survey would be to list the places you have visited in the last five years and then rate them on a scale of 1-10 with 10 being the most overrated. Then you could continue to build your pool until you had enough data to fill it.

Consumer Prowess

Even though the typical consumer today is smarter than any of their predecessors, their attention spans are so short that they live and die by the titles of articles and the bullet points within. They don't have the time to read a two-paragraph review to find out why the restaurant was bad, they just notice that it only got two stars so they better not go.

They don't know why a city has great shopping, they just know that there were three surveys that they found online that said, "Atlanta had the best" and so that is where they are going to go.

It makes sense that marketing departments for corporations are always going to put their company's face in front of any stats or polls that they put out. In order to fix the system, the independent companies have to be the mediators; the referees. It is up to them to seek out real sample sizes, deep understanding and factual statistics to back up their headlines. Because as heavily read as Yahoo is, if I was Las Vegas, New York or Miami, I would be very concerned about the effect of a poorly conducted poll on my tourism numbers for this year.


For the latest travel news, updates and deals, subscribe to the daily TravelPulse newsletter.

Topics From This Article to Explore

Tom Bastek

Tom Bastek

Tom is a writer in the Atlanta Area. He has traveled to all 48 continental states and almost every stop in the Caribbean. In...

Get To Know Us Better

Agent At Home

Helping leisure selling travel agents successfully manage their at-home business.

Subscribe For Free

Agent Specialization: Group Travel

Laurence Pinckney

Laurence Pinckney

CEO of Zenbiz Travel, LLC

About Me
Agent At Home

Helping leisure selling travel agents successfully manage their at-home business.

Subscribe For Free

Agent Specialization: Group Travel

Laurence Pinckney

Laurence Pinckney

CEO of Zenbiz Travel, LLC

About Me