Big Changes Are Ahead For The Health Care Industry, Courtesy Of Big Data

The importance of Big Data transcends its big hype.

Illustration By Alex Eben Meyer

Just about every week, another startup announces plans to mine the world's mountains of information for meaningful insights. There are so many blue-sky proclamations for what's become known as Big Data that you need a data scientist to track them all. Splunk, the first Big Data star to go public, surged to a $3 billion valuation when it hit the Nasdaq in April, despite just $121 million in 2011 revenue. The hype has already provoked a backlash, but you'd be foolish to discount the power of analyzing large tracts of data.

Most Big Data conversations take place in the abstract or focus on the horse race of who will dominate this market. But if you want to understand why there's excitement about Big Data, drill down into an example. A few years ago, at the Washington Hospital Center, a sprawling urban health-care facility near D.C., the emergency-room doctors noticed a troubling medical msytery common to many hospitals: A large number of patients were returning to the ER, sick again, within just a few weeks of being discharged. Doctors believed that many of these readmissions could have been prevented with better follow-up care, but they had no way to predict which patients were most likely to become ill again. The issue became even more pressing when Medicare announced that it would start penalizing hospitals for patients readmitted in less than 30 days after discharge.

SPLUNK Makes software that collects data from around your company, then dishes it up in any way you please.

IBM Big Blue is investing $100 million to expand its expertise in data analysis and has acquired several data startups over the past year.

PALANTIR Started by former PayPal whiz kids who helped the CIA track down terrorists, Palantir makes software that finds connections in large, disparate sets of data.

WAVII Wavii crawls and analyzes the web to build an interactive corpus of everything.

METAMARKETS A dashboard for all the data in your firm; currently being used by many online media sites to price and serve ads.

At a loss for answers, the hospital began working with an unlikely Dr. House--Eric Horvitz, a computer scientist (and physician) who works at Microsoft Research. Instead of a team of slavish residents, Horvitz's main investigative tool is data. He and his colleagues built a system to analyze more than 300,000 ER visits. They looked for correlations among 25,000 variables, including the patients' medications, vital signs, and doctors.

In the thicket of information, Horvitz stumbled upon a few intriguing surprises. One was the length of a patient's stay in the hospital. Horvitz's data showed a tipping point of 14 hours--if you're in the ER for longer than that, there's a good chance you'll be back. Another red flag was fluid. The data showed that any mention of that word on a patient's medical chart significantly increased the likelihood of readmission. "You look at that and you say, 'Wow,' " Horvitz says. "We don't know why fluid is so important, but it seems to matter--and it can help doctors take necessary preventative action."

Horvitz built his findings into a program called the Readmissions Manager, which Microsoft sells as part of its health-care IT suite. It analyzes hospital data to produce a readmissions forecast for each patient, and doctors can tailor follow-up appointments based on the forecast. How much will this reduce costs? Horvitz is formulating clinical trials now to assess the savings. Studies show that 20% of patients released from American hospitals need to be readmitted within 30 days, an inefficiency that costs Medicare $17 billion a year. "There's just so much data dropped on the floor in health care that we can learn from," Horvitz says.

And it's not just health care. Most of the world's large organizations are now both incomprehensibly complex and obsessive about tracking what they do on ever-cheaper racks of servers, which means that Big Data is emerging as a tool in finance, law enforcement, advertising, customer service, and pretty much every commercial transaction on the planet. "Many businesses today walk around like amnesiacs," says Mike Driscoll, CEO of Metamarkets, another analytics startup. "Some of this will take longer to realize than others," he adds, a point echoed by every Big Data player. Which is why, long after the hype subsides, Big Data will still be a big idea.

A version of this article appears in the July/August 2012 issue of Fast Company.