What Are We Really Measuring?

As anyone unfortunate enough to have shared space with me over the past week can tell you, I’ve been reading Malcolm Gladwell’s The American Way of Killing. It has consumed me in a way I didn’t expect, and one of the ideas that has stayed with me is how the statistics we use to understand a problem don’t always tell the whole story.

In looking at gun violence in America, Gladwell points to an interesting disconnect between what the statistics show and what they might actually mean. A declining homicide rate would seem to suggest that fewer people are experiencing deadly violence. But what if there’s more to it than the raw numbers? Could it be that advances in emergency medicine and interventions have made it possible for more people to survive injuries that even just a few years ago might have been fatal? In other words, a decline in deaths doesn’t necessarily mean a corresponding decline in violence.

It’s a good reminder that what we measure (and what we don’t) shapes how we understand a problem. And this got me thinking about the nonprofit sector.

We spend a lot of time measuring our work. We count the number of people served, programs delivered, dollars raised, and volunteers engaged. We track outcomes, report progress, and share our results. These numbers matter. But how often do we stop to ask whether what we’re collecting is telling us what we really want to know?

An organization might celebrate serving more people this year than last. Does that mean that the need is growing? That the organization is reaching more people who need help? Or maybe both?

A fundraising campaign might exceed its goal. But does that success reflect a growing donor base, or was it driven by a few unusually large gifts? Are more people investing in the mission, or is the organization becoming increasingly reliant on a smaller group of donors?  What might those answers tell us about the long-term sustainability of the organization?

The numbers matter, but so does our willingness to look beyond them, to question our assumptions, and to remain curious about what they tell us. And what they don’t.

Sarah Plumb, MA, is a consultant with The Rome Group, specializing in grant writing and communications.