Earlier this week, STELLAService posted a blog entry that summarized airline customer service on August 26, the day before Hurricane Irene began seriously impacting the East Coast. The group utilized phone hold times, Twitter responses, and Twitter response times as metrics to measure customer service.
STELLAService’s idea is a perfectly valid one. Having independent third parties look into customer service for any industry is often a positive for consumers. Unfortunately, the group’s methodology for some airlines appears to be a bit…flawed, especially when it comes to Twitter.
Based on some searching, it appears that STELLAService utilized a bunch of Twitter accounts to perform their experiment. While STELLAService would not confirm what questions they sent, some tweets sent while the study was taking place seem quite suspicious. As the screenshot below indicates, there were four identical tweets sent to airlines within three minutes of each other.

This question, if it is indeed one posted by STELLAService, is dubious at best. The number of passengers connecting in South Carolina is probably small. Meanwhile, JetBlue, and AirTran received this question, even though none of these airlines fly to South Carolina.
There are some other issues with the study. Note that as part of the study, tweets were sent to @aairwaves and @continental, both of which are defunct Twitter accounts. (This fact is noted on said Twitter pages.) AirTran only has one tweet from 2009, though the Twitter page says nothing to indicate if someone in monitoring it or not.
Questions aside, there appear to have been some issues (in my opinion) with how the results were reported. Here’s what STELLA originally reported:

What’s also interesting is that the study claims that United/Continental didn’t reply to any tweets. Apparently United did reply to the tweets sent to the @continental account, but used the current @united handle instead. (That seems perfectly logical – one would want to move customers to the new account.) Questions sent to the @united account did not receive a response, apparently, so STELLA just reported that, making United’s results look worse than reality.
STELLA has since updated its chart to note that United replied to queries. Meanwhile, American was completely absent in the updated chart. The group noted that “the tweets sent to American Airlines were sent to an account they deem to be inactive, so we have removed their Twitter findings.”

As a data junkie, I love any studies that try to add more insight about the airline industry, but this study examined social media response times is a bit lacking. First, if I’m right about my guesses about the tweets, some identical questions were sent to airlines. That’s not a huge problem, except when said questions aren’t applicable to some carriers. Second, defunct Twitter accounts were used in the study, potentially skewing results.
That said, not all of the results seem that far off. It’s not shocking to see Delta, Frontier, and JetBlue in the top three, for example, considering the investment those carriers have placed in social media. And I do look forward to more studies like this, especially those that incorporate the responsiveness of airlines over social media channels.
EDITED on 9/5 — I had incorrectly stated that Spirit did not fly to South Carolina.
Latest Comments