by Joe Miller,
Director of Development
Two weeks ago, I attended the 105th Annual Meeting of the American Meteorological Society in New Orleans. It was my first time to both the AMS and the Big Easy. I’m not really a weather person, but I have been working on a project with the National Weather Service. I was at AMS to talk about that work.
The conference is huge—more than 7,000 registered attendees gathered in a conference center that covers two city blocks. The program listing presentations ran close to 100 pages of small print, and talks covered a bewildering array of topics. It was a good reminder of my biggest takeaway from working with the weather service.
The weather enterprise is almost incomprehensibly complex.
Take something like flash flooding—a weather disaster that is all too familiar to those of us in Pocahontas County. To predict floods, you need to know how much rain will fall. But you also need to know its intensity—two inches of rain over a day is very different from two inches in an hour— and you need to know exactly where it will fall.
Forecasters pinpoint weather location by dividing the entire country into a grid of 2.5 km (about 1.5 mi) squares. For context, the town of Marlinton is around 6 square kilometers, so it would fall into three grids. The contiguous United States contains over two million of these squares.
But knowing the rainfall intensity and amounts for each grid point still isn’t enough information. Flooding also depends on how wet the ground already is, how much water is already in each waterway and at what level each of them floods.
The surrounding geography will determine what happens to floodwaters, so forecasters need to know that. And we all know that flooding will change the surrounding geography—the next flood won’t look like the last one!
Forecasting flooding involves analyzing literally millions of data points.
Figuring out how to make that information useful is an ongoing problem for forecasters and meteorologists.
How do you communicate millions of data points?
You could imagine a really big spreadsheet that shows all the weather data for each grid point. Your file would have two million rows and several dozen columns. It also wouldn’t help anyone understand the weather. You couldn’t even open the file without special software.
But if you plot that data on a map you get the National Forecast Chart, which is the first thing you see on the homepage of the Weather Prediction Center’s website (www.wpc.ncep.noaa.gov/).
Or if you take advantage of the fact that smartphones know where you are located, then you can build a weather app that shows you data from exactly the right set of grid points.
I call this sort of thing shifting complexity.
A map of a forecast still contains several million pieces of data. But the individual data points are hidden—the complexity is handled by a bunch of algorithms that plot the data onto the map.
The same is true for the various watches, warnings and advisories that the National Weather Service issues. There’s a lot of complexity behind them. You’re getting the big picture, not the details.
Of course, there’s danger in shifting too much complexity. Weather forecasts sometimes show a low probability of life-threatening events. If you put too much emphasis on the low probability part, lives may be lost if the event happens. If you over-emphasize the life-threatening part, then you lose credibility if the event fails to materialize. Finding the right balance of complexity is crucial.
I help the weather service test its products to find the right level of complexity.
Librarians also shift complexity.
Caroline and I spent some of our time in New Orleans visiting bookstores. The last one we visited was – well, it was A Lot. There were thousands of books piled on shelves, in stacks on the floor, in boxes—all with no organization and, in many cases, no way to even read the spines.
I showed a photo to my colleagues. “Total madness” was the least horrified reaction.
Librarians organize the “total madness” of information into something usable. Every book in the library has hundreds of data points (metadata) that librarians catalog. The books are organized on shelves according to that metadata. But you don’t have to know all that information to use the library—just a couple of data points will get you where you need to go.
The complexity is shifted from patrons to librarians—which means you can just walk into a library and find a great book to read.
