It’s Christmas Day here at Boomerang HQ. Since nobody’s in the office, for the moment, Boomerang HQ is technically my dining room table.
I’m the only one from the Boomerang team working today, and I’m spending just 45 minutes making sure urgent customer support emails get answered before heading off to spend the rest of Christmas with friends and family. (Yes, this post was prewritten!)
Nobody was in the office yesterday, either. Unlike some other startups who treated working during the holidays as evidence their company was in it for the long haul, we’ve been doing this for over five years. We’ve already been through a long haul. One of the things we’ve learned is that unplugging during the holidays and during vacations is essential when months turn into years.
Handling our Christmas Day support is a tradition for me. Every year since we shipped Boomerang in 2010, I’ve claimed that day for handling support during the holidays. It earns me a lot of credit with our team — if the CEO cares enough about our customer experience to answer support emails on Christmas (and cares enough about the team’s work-life balance to make sure that nobody else has to do so), then everyone should care a lot about our customer experience.
For each of the last five years, I’ve always found spending a half hour to an hour communicating with our customers on Christmas satisfying; even enjoyable. That’s unusual, because when I help out with support on other days, I often get frustrated. After five years, my identity is too interwoven with the products and the company, so I take every unhappy email personally.
This year, we finally have data to show why answering support email is likely to be such a pleasant experience for me today — it turns out that taking support on Christmas Day might get me a bit more credit than I deserve. In addition to a much lighter than usual email load, we now know definitively that customers are unusually cheerful and forgiving when web services don’t work right on Christmas!
On this year’s big team workaway (to Maui — yes, we’re hiring), we decided to spend a couple of days trying to understand what properties make an email effective. We spent plenty of time training neural networks and evaluating natural language processing algorithms, but one of the most significant factors was one of the simplest to calculate — the sentiment of the writing in the email. When testing those algorithms, one of our best real-world data sources was our own customer support email history.
Press enter or click to view image in full size
Looking at customer support data through the lens of sentiment gave us a different perspective than when we’re answering it each day. As you might expect, we found some real gems. Though oddly poetic, this was the most negative customer email we’ve ever received (looks like the sentiment analyzer generally worked!)
SO SICK OF YOUR CONSTANT INTERRUPTIONS.
STOP ASKING IF I WANT YOUR FUCKING PROGRAM, BOOMERANG.
I DON’T.
But by and large, our customers had much more good things to say than bad. Emails with extremely positive sentiment outnumbered messages like the one above by 4-to-1.
I absolutely love the new features to manage “Boomerang” e-mails within GMail. I tripped on the new functionality a few days ago and thought “huh…I think that’s new…and I love it”
To those engineers who endured the dark room, likely fueled by pizza and 55-gallon drums of Mountain Dew, thank you!
Thank you, too, Matt!
It turns out that 99.2% of days had more positive emails than negative — in five years, we only had ten days where the overall tone of our support email was actually negative. I really like our customers. Thank you for being awesome!
We wondered if there would be any subtle trends in the data — for example, we compared the performance of the Standard and Poor’s stock index to the sentiment of our support emails.
Press enter or click to view image in full size
Not much to see here, huh? We also checked to see if support sentiment might be a lagging indicator for stock market performance or, perhaps, if it might end up a leading indicator (new business model — hedge fund!), but they were equally inconclusive.
When we went through a few specific dates where the sentiment was an outlier, some trends became clear, though.
It turns out the most significant factor, by far, for determining a bad customer support day was service downtime. During our workaway to Iceland in the summer of 2014, we got caught with an unannounced change to Google’s IMAP support — a new cap on the maximum tree height for IMAP search queries. Boomerang was completely down.
Between inconsistent internet access, being in a different time zone than virtually all of our customers, and the technical challenges involved, it took us almost four hours to get it fixed. By comparison, the only downtime Boomerang experienced in all of 2015 was one instance of scheduled maintenance for migrating our database hardware. Understandably, our customer support email was the least-pleasant it had ever been that day.
On the other hand, the best predictor for a great customer support day is a holiday. Virtually every holiday was in the most positive 10% of customer support days. The absolute most-positive day we’ve ever had from a support perspective was a Monday, July 8th. Since the American July 4th holiday fell on a Thursday, a lot of folks must have really enjoyed a 4-day weekend during the warm summer days.
In the same vein, two out of our ten most positively-sentimented support days of all time have been on Christmas Day. Intermingled with a few issues that need untangling (mostly written very politely, and signed off with Happy Holidays or Merry Christmas) come holiday greetings and thank you notes. Looking back through some of the messages, they are overwhemlingly cheerful. Combined with a mug of hot chocolate and the smell of my Christmas tree, answering emails when a majority of them could legitimately be classified as tidings of comfort and joy isn’t a bad way to spend an hour today.
One other note: on Christmas Day in 2012, we set up an autoresponder to let customers know that our responses might be slower than usual. The sentiment on that day was by far the lowest of any Christmas Day we’ve had. We haven’t used one since.
Taking a quick tour through the news the last few days, I’ve seen dozens of articles about shopping nightmares, family drama, cooking disasters, and every other holiday problem that could arise. You’d think that the holidays just make us stressed and unhappy, and we’d be better off without them. But the sentiment of emails sent on Christmas says otherwise. These emails, sent to an anonymous role-based email address, usually to get help with an issue, show that the holiday season really does make us more cheerful, nicer people.
Merry Christmas!