Thanks for mentioning the paper that Bruce and I wrote. I wanted to share some approaches to “measuring impact” that we have been trying with a foundation that is seeking to catalyze a network of nonprofit leaders in a major U.S. city by creating the conditions that transform the relationships network members have with each other, and their relationship to the greater good of the community. We are experimenting with tracking several different kinds of data.

• Changes in positions and roles (job changes, serving on each other’s boards, joining the boards of school systems, community foundations, or other entities that influence the quality of life for the city and the nonprofit sector)
• Changes in the pattern of network connections (bonding, bridging, clusters, shared activities, goals and interests)
• Stories of emergent collaboration, cooperation, and coordination (those stories that are directly linked to relationships that were catalyzed, transformed or deepened through participation in the network)

Each of these requires different methodologies for gathering, interpreting, and presenting the data in order to make a compelling case for impact. I highly recommend that people take a deeper look at Lawrence CommunityWorks and how they are evaluating their impact. Net Gains which is mentioned in your blogpost discusses in depth the evaluation of the Lawrence CommunityWorks network. This piece and others by Bill Traynor who is executive director of Lawrence CommunityWorks can be found on the Leadership and Networks Wiki page http://www.leadershipforanewera.org/page/Leadership+and+Networks.

One challenge I have found is that network sponsors often have too short a time horizon for their investments. They also under-invest in reflective learning and data collection over time to tell the story of impact. I believe the story of network impact can and should be told in multiple ways, each of which may be compelling for different audiences. Having said that my colleagues and I with the Leadership Learning Community have also found that the question of ultimate value, e.g., was this worth the investment, can only be fully answered by the network (and its sponsors) taking stock of their values and purpose, and deciding for themselves how valuable it is for them and the community.

Thanks for this useful summary Diana. One of the stumbling blocks I have found in our collective learning process is the desire to jump right into a “productivity” network (here I am using the Plastrik-Taylor language). Funders and practitioners often want to jump there because it is the more familiar land of “measurable outcomes.” But leaping too fast can actually limit the potential of a network to actually innovate as well as it’s capacity to become a malleable structure that can easily re-organize itself for different purposes. I realize that I am not offering much in terms of how to measure, but I do hope I can call attention to the idea that networks can have an explosive potential and that rushing them into production land can force us into technical applications and away from adaptive possibilities.

Claire – thanks so much for the thoughtful post! It’s all really helpful — and especially instructive to call out the cases, your experience with the leadership network and LCW as a model of network organizing that is being intentional about how their assessing their impact.

Gibran — I agree wholeheartedly. It brings me back to the purpose discussion. The tendency is, as you say, to assume the purpose must be about ‘production’ – doing something together that results in measurable outcomes when, in some cases, the purpose may be simply building connectivity so that the network (or subsets) can act together when the need / opportunity arises. It does raise the question: how do you measure investment in self-organizing capacity?

Kudos to Diana for a thoughtful summary! Making a case can usefully rely on history. Moreover, network weaving isn’t a newfangled idea (as some social media types tend to think).

We have funded professional societies, for example, in conventional activities like journals (the spine of a network) or recruiting (e.g., of minority environmental scientists). A conventional summative evaluation should have provided data at the end of the grant about whether a network had grown, along with other measures (participation, leadership, etc.)

More broadly, much social action succeeds only if a network forms. Networks and formal organizations are the essence of institutionalization, after all. If the networks themselves are elusive (“dark matter”), one can infer their presence through observations of innovation and diffusion (“quasars”). Again, retrospective analysis can be recruited to this purpose.

Oliver Williamson’s Nobel prize in economics this year was won with analysis of why informal or market coordination (much of which is networks) is sometimes inferior to formal organization. So there’s a body of thinking out there that may be worth diving into.

My impression is that drawing together information along these lines would make the case for networks. But the harder question is to gain insight into the appropriate balance in funding. How much should one invest in networks vs. organizations. There, some work along the lines of Williamson’s (which I do not profess to know) may be useful.

Kai – thanks for this. Working with the funders community of practice, we’re hoping to take a look at theories for change for networks. It’d be instructive to zoom out to the level of program strategy to better understand the role of network (and org) investment.