iStock / filo
Yes, many people think evaluation is a chore. But done right, it can be an astonishingly effective tool in building excellent ministry, writes the director of a Youth in Theology and Ministry program.
I love evaluation. Yes, that’s right. I love evaluation.
I realize that many people in theological circles would equate program evaluation to St. Paul’s “enduring all things.” But I think we should instead consider it a “more excellent way.”
Paul might take exception to relating evaluation to the more excellent way of the spiritual gift of love. But perhaps one could consider that providing excellent ministerial programming is a way that pastoral leaders show Christ’s love.
For 35 years, I’ve heard colleagues complain that evaluation is basically hoop-jumping to appease others. And I understand this — many theological leaders don’t see the benefits of evaluation.
But I love evaluation, because I’ve seen how it can be an astonishingly effective tool in building excellent ministry. I want to share four steps to developing a “more excellent way” of ministerial leadership through evaluation.
Step 1: Think strategically
When beginning an evaluation process, two questions arise: “What should be evaluated?” and “How will this help?” Answered thoughtfully, with a clear understanding of the context and needs of those being served, these questions can lead to deeper, strategic thinking about program goals and outcomes and about how the evaluation data might be used to improve the program.
Excellence in ministry demands strategic thinking, especially as ministerial leaders take on more responsibility.
This might seem obvious, but I’ve found that many pastoral leaders resist strategic thinking. The Gallup Organization’s research provides some insight here; the book “Strengths Based Leadership,” drawn from their findings, suggests four domains of leadership strength: executing, influencing, relationship building and strategic thinking.
Different leaders have different strengths. Pastoral leaders, who are often quite good at fostering relationships with people, are often lacking in the strategic thinking domain. To achieve organizational excellence, these relational ministers must seek out team members with complementary strategic thinking strength.
Step 2: Articulate goals and outcomes
Successful evaluation always begins with goals. If a theological program does not have clearly articulated goals and outcomes, then there is nothing to evaluate. Stephen Covey got it right: “Begin with the end in mind.”
Last summer, I taught a practical theology course on evaluation. I asked the graduate students who primarily served as catechetical leaders in congregations whether their programs had written goals and outcomes. A few said yes, but many of the students stared blankly at me.
Then I asked whether they used published curricula. Most said yes. But when I asked whether they knew the learning outcomes for those published curricula, their clueless eyes suggested the quote from Yogi Berra, “If you don’t know where you are going, you might wind up someplace else.”
Theological leaders will absolutely end up somewhere else without clearly articulated goals and outcomes, so defining them is a crucial step in the evaluation process.
Step 3: Develop tools to assess results
How can a pastoral leader know whether a program’s outcomes are achieved? It’s important to ask this question before programming begins. Unfortunately, we often employ very limited measures of success.
When I was a youth minister, the only evaluation question I was ever asked was, “How many youths showed up last night?” Participation is important, but Christian formation is about much more than numbers. Yet that’s all I measured.
In my current role, I have the privilege of working with the directors of seven seminary- and college-based youth theology programs. They are part of the Youth Theology Network coordinated by the Forum for Theological Exploration (FTE) and initiated by Lilly Endowment Inc.
In planning the summer youth theology institutes, we met with an evaluation consultant to identify common outcomes. We then developed surveys to give to participants before and after the institutes.
One desired outcome was “The youth will experience Christian community.” To measure this, we would ask participants to agree or disagree with the statement “I have been part of a vibrant (lively, animated, vital) Christian community.”
Another desired outcome, “The youth will grow in their excitement for theological learning,” would be measured through their response to “I am considering theological study in college.”
These relatively easy-to-create surveys would help us measure whether and how the participants changed as a result of the programs.
Step 4: Ponder the data and make adjustments
When we looked at the pre- and post-institute survey results from approximately 200 youth in the summer of 2019, we got some good news: the number of youth who strongly agreed that they had been part of a vibrant Christian community increased 60%. The number who strongly agreed that they were considering studying theology increased more than 90%.
But evaluation data aren’t always positive. In 2019, the youth theology program that I direct, Youth in Theology and Ministry (YTM), had an alarming decline in participants’ assessment of their positive experience of community.
For nearly two decades, 90% of youth participants responding to the statement “Assess your overall experience of community at YTM” had said “excellent.” In 2019, only 61% said “excellent” — the lowest in 19 years.
Something was wrong. Reviewing and analyzing all the data, we discovered that they reflected our failure to understand the complexity of developing Christian community within a diverse population of youth.
In 2019, YTM had achieved for the first time a population of youth that was 50% persons of color. The evaluation question helped us realize that racial and ethnic barriers could affect the development of community in ways we did not understand.
As a result, the YTM staff committed to learning more about intercultural ministry and effective strategies to deal with intercultural conflict. For example, we recently attended a lecture and workshop on intercultural ministry by Arturo Chávez, the president and chief executive officer of the Mexican American Catholic College in San Antonio, Texas, and we’ll be applying this new knowledge to next year’s institute.
Without that survey, we might not have realized that something had shifted and that we needed to make changes to this long-standing program.
That’s why I love evaluation. It provides insight into what is working and what isn’t — and challenges a team to address the issues that arise.
Most of all, I love evaluation because it is a window into a “more excellent way.”
