NOTE: This post is to introduce you to the 18th episode of François Matarasso’s and my monthly podcast, “A Culture of Possibility.” It will be available starting 17 June 2022. You can find it and all episodes at Stitcher, iTunes, and wherever you get your podcasts, along with miaaw.net’s other podcasts by Owen Kelly, Sophie Hope, and many guests, focusing on cultural democracy and related topics. You can also listen on Soundcloud and find links to accompany the podcasts.
Every third podcast we skip having a guest and talk about something François and I choose. François inspired me to suggest this month’s topic when he posted an essay entitled “This is not a level playing field,” describing his “worry that current approaches to evaluation in the arts are misguided and even damaging.” I agreed with all that he wrote and suggested I could add a few observations about the evaluation situation in the U.S.
On the podcast, François set the scene by talking about the reasons to evaluate work. “When I started working in community art in 1981, evaluation wasn’t an issue at all. I don’t think it even crossed our minds. Occasionally you’d have to report to a funder on what you’ve done. You’d try to invite them to come and see.” A different approach to evaluation came “in the late 80s as part of the whole new public management element of the neoliberal agenda. Making public services accountable, supposedly, although I think, curiously, nobody seemed to think that private sector services need to be made accountable for what they were being paid for. But that’s another story. So I’ve been thinking and working on evaluation now for 30 years, and I’ve watched its development and the essay was really born out of growing frustration.”
“You can only evaluate something against its purpose,” François said. “I think the purpose of community art is to make article 27 of the Universal Declaration of Human Rights more of a reality than it is. It says that everyone has the right freely to participate in the cultural life of the community, and to enjoy the arts. It doesn’t say that you have to do that in order to be a better person, or a more civilized person, or a less troublesome person. It just says you have the right to do this.
“I’ve always said that evaluation is integral to creative practice. There’s no problem with evaluation as such. The problems arise from evaluation becoming part of a contractual arrangement with a funder to deliver things that I think either can’t be or shouldn’t be delivered by community arts projects.” François and I had both heard from many people who were required to say how their work had changed participants, to reduce that to numeric factors such as impact on grades, test scores, or drop-out rates, who were thus forced to evaluate their work according to aims that were not integral to participants’ own reasons for joining.
We both agreed that many funders’ evaluation demands are becoming more complex, lengthy, and onerous, in the U.S. and in Europe. François saw the problem as particularly serious in the public sector, whereas in the U.S., certain private funders’ demands seem more problematic.
“Evaluation,” said François, “will grow to fill the space you give it. I’ve done quite a lot of helping other people to evaluate their work. You can ask all kinds of questions. There’s almost nothing you can’t know if you’re prepared to pay the price of finding it out. And that is not only a financial price, but in terms of time invested and interference in the process. People who commission and pay for and fund art projects seem to be entirely oblivious to that. It’s really easy to think of more things you’d like to know the answers to if you’re sitting in an office somewhere, and then you turn that into more expectations, more monitoring requirements, more data gathering. People literally spend weeks writing their grant applications and submitting them, then they get to do the work, and then they have to spend more weeks writing up and reporting on what they’ve done.”
But it isn’t the same for everyone. I pointed to “the differential way that this expanding evaluation addiction is applied. The symphony orchestra, the ballet company, and the museum are not being evaluated in the same way on the same criteria. Even though they represent expenditures of public and private funds, and people want to know that there’s value for money, the fact is, they don’t have the expectation that somebody’s going to be transformed by buying a ticket to the ballet. So they don’t ask all those questions about how the money is curing social ills. The evaluation stuff that I’ve seen from major institutions is mostly audience data: how many people came, that kind of thing. That is very, very unfair, because as you pointed out, it places additional requirements on those who have fewest resources.”
François observed that the same evaluation requirements land very differently as burdens if you’re an institution with a staff of 50 or a community group with a staff of three, and if you’re a community arts group versus a major institution. “Ballet companies, orchestras, theaters, galleries are not required by their their sponsors and donors and funders to provide that kind of data because they have an existing cultural legitimacy. In the view of the art world, if you choose to go to a gallery or to the ballet, then you’re demonstrating that you’re already a civilized person. So there’s no need to improve you. It’s only the poor, who have to demonstrate in order to get access to any funding that might guarantee their right under Article 27, to participate in the cultural life of the community. They have to demonstrate that somehow they have been improved by their contact with culture. And I find that implicit idea that some people need improvement through culture and others don’t so obnoxious.”
That inspired me to talk about what I call “institutional narcissism,” funders wanting to look good. “We want to maximize our ability to make grants that are considered blue-ribbon grants. We want to be in a protected position by framing what we do as evaluation as kind of a neutral practical tool that we use to make the best possible grants.” In the U.S., especially in the private sector, there are many rejected applications for every grant that’s approved. “So funders are looking for a culling mechanism, making people aware that they need to jump through a large number of hoops to be qualified, to understand those hoops and how to navigate them and express it in the way that funders want. A lot of applicants are going to fall by the wayside, discouraged by what it takes to apply, and that’s going to reduce funders’ burden. Carried over to the evaluation frame, it’s going to give us a basis on which to say, ‘Well, that didn’t really work.’ And it’s going to seem objective and quasi-scientific. And that’s going to help us out because then we don’t have to answer to charges of preferences and favoritism.” But actually what’s being expressed here a preference for the major institutions, giving them the most money because their legitimacy is intrinsic and settled.
François thought that sounded a little more deliberate than his take. He brought his experience with funding programs into the conversation, describing ways to evaluate that avoid wasting people’s time. He pointed out that funders could reduce the number of applications by articulating their criteria with much more focused specificity. And we both recognized that a subset of funders has been keen to avoid wasting applicants’ time, streamlining the process so the first step is a simple letter of inquiry. I just wasn’t sure that leveled the playing field much. Listen and decide for yourself!
With that beginning, we touched on the trend for foundations to switch from grantmaking to operating their own programs to control outcomes even more; exactly what is wrong with the concept of “best practices;” the role that risk aversion plays in funding; whether there’s anything to be learned from other types of investment; how nonprofit funding has been distorted by corporate models; the underlying class biases that shape funding; and how these problems are structural, affecting the sector regardless of how conscientious and well-intentioned the individuals running programs may be.
“The Judgement” by Solomon Burke.