Evaluation: The Art of Arts Fellowships (Summer 2004 Trust Magazine article)
Each June, the Philadelphia art world, and beyond, pays attention to the announcement of artists recognized by the Pew Fellowships in the Arts (PFA is a project of the University of the Arts)--partly because the award is substantial and prestigious, and partly because of the excellence of the Fellows chosen since PFA was established in 1992.
PFA is one of the Trusts' seven Artistic Initiatives, which were designed collectively to improve and expand the caliber of local cultural programming. PFA contributes to this overarching goal by offering substantial financial and professional support to a wide range of individual local artists. Applications for fellowships are accepted across 12 artistic disciplines and are reviewed by panels of distinguished artists and arts professionals from outside the region. Those chosen receive $50,000 grants to enable them to dedicate themselves full-time to artistic exploration and production.
During PFA's 10th anniversary, the Trusts' Culture program and Planning and Evaluation department embarked on an evaluation to examine whether PFA was meeting its programmatic objectives and, with this knowledge, to determine what refinements might make a stronger program. The Trusts had three goals for the evaluation: to understand the impact of PFA on its fellows, who numbered 174 as of June 2004; to assess the "value added" to both the local arts community and Philadelphia as a whole; and to hear what changes would improve the program's impact on the Fellows and the Philadelphia arts community.
To perform the evaluation, the Trusts selected Dvora Yanow, Ph.D., professor and chair of public administration at California State University at Hayward, an expert in qualitative research methods and evaluation of organizational impact and effectiveness. Yanow conducted in-depth interviews with fellows from every cohort, 85 in all (a response rate of 63 percent among those she was able to contact). She also interviewed 32 local and national cultural leaders, PFA applicants who were not chosen as fellows, and past members of PFA selection panels. She supplemented information from these interviews with an analysis of PFA reports and publications, prior evaluations and other relevant documents.
KEY FINDINGS
Programmatic and Operational Strengths
Inclusiveness
Fellows and cultural leaders alike complimented PFA for awarding fellowships in 12 disciplines: choreography, craft arts, folk and traditional arts, fiction and creative nonfiction, media arts, music composition, painting, performance art, poetry, scriptworks, sculpture, and works on paper--a reach that is comprehensive and encompasses newer forms of expression.
Because fellowships are awarded in three disciplines each year, each cohort group meets artists in other fields, leading to exchanges of information and sometimes collaborations. Fellows appreciate the application process, which is open rather than nomination-based, and available to artists at any stage of their career.
Flexibility
Fellows determine when to begin their fellowship and whether they want to extend the award over one or two years. They are not held to predetermined deliverables at the end of their term, but instead may use the fellowship to pursue their artistic development wherever it leads. Fellows say that PFA "treats its award recipients as adults" by recognizing the unique and unpredictable nature of the creative process.
Substantial Award
The $50,000 award is one of the largest fellowships in the country. PFA has awarded $8.6 million since its inception.
Rigorous Jury Panel Selection and Management
PFA selects distinguished artists, curators and arts leaders from outside the Philadelphia region to serve as panelists who choose the fellows, and their own reputations lend the program credibility and prestige. Because panelists live beyond the local area, judging is perceived to be objective and fair in what could otherwise be construed as a very subjective process. Former panelists commented that they had never before been so well prepared for a panel process.
Arts leaders acknowledged the clarity of the criteria by which applications are judged. In the words of a museum executive, "The ideals of wanting to support creativity [are] front and center."
Effective Administration
PFA staff--Director Melissa Franklin and Program Associate Christine Miller--were repeatedly singled out for an outstanding job of directing and implementing the program, creating a presence for PFA in the Philadelphia region, representing PFA to a wide public and interacting with artists and the arts community locally and nationally.
Impact
On Fellows
By enabling artists to devote significant time to their creative work, the fellowship program has positively affected the recipients' artistic development. Fellows reported being able to experiment with new techniques, take on larger or more complex projects, work with new media, collaborate with other artists and build their portfolios. In some cases, this enhanced productivity resulted in sales, commissions and gallery representations.
Fellows' subsequent honors are noteworthy and speak to the quality of artists who have received the award: for instance, orchestra commissions, performances in the Lincoln Center Jazz Series, participation in the Whitney Biennials, solo shows in New York City, the National Treasure Heritage Award and fellowships from the John D. and Catherine T. MacArthur Foundation and the John Simon Guggenheim Memorial Foundation.
On Culture in Philadelphia
PFA has contributed to the vitality of Philadelphia's cultural community by increasing innovation, production and efforts across disciplines and subsequently helping fuel a broader and higher-quality array of artistic offerings. As described by one national arts leader, "All artists, whether they get money in their bank account or not, are benefiting from the extraordinary programming supported and enabled by Pew. [PFA] has a broader impact than just on the individual level."
Through a listserv, a semi-annual newsletter and www.pewarts.org, PFA makes fellows aware of the work of their peers in the Philadelphia area. Former Fellows have noted that the PFA enabled them to explore arts beyond their own discipline and establish collaborations with other artists, some of them fellows themselves.
Arts community leaders mentioned that PFA contributes to the trend that is keeping local art schools' graduates in the city; they want to stay and be part of this vibrant arts scene and, in the meantime, fulfill the two-year local residency needed to become eligible for the PFA.
On PFA and the Trusts
Local and national arts leaders and grantmakers regard PFA as a well-designed, well-administered initiative that has contributed to the increase in high-quality, diverse arts offerings in Philadelphia. National funders especially noted the objectivity and rigor of PFA's selection process and the use of nationally prominent panelists.
Early on, evaluation participants referred to PFA as "on the cutting edge" and "ahead of its time." It should therefore come as no surprise that PFA has been perceived as a model for other arts-awards programs. In Philadelphia, several local funders described the Trusts' support for PFA as a factor in helping legitimize their work; in particular, these grantmakers reported that PFA's success strengthened the case within their own organizations for investing in individual artists.
RECOMMENDATIONS
Strengthen Identity
PFA creates a cohort of artists from a geographically bounded region, enables them to step away from their normal work schedule (e.g., teaching) and brings them together in a common situation that offers a nascent identity. Yanow recommended that this aspect of the fellowship be strengthened, especially since the award reinforces the artists' confidence and provides them with a sense of community in an otherwise isolating profession.
In addition, by providing more opportunities for fellows to come together, PFA could encourage them to interact with one another, mentor newly appointed fellows and serve as informal program ambassadors. Cultivating the fellows' identity could be accomplished through regular gatherings, and through various communications efforts.
To this end, PFA has implemented a semiannual newsletter that features articles focusing on past and present fellows and listings of their activities, such as productions, exhibitions and publications.
Enhance Professional Growth
To ensure that the fellows get the most out of their award, Yanow recommended that PFA draw upon former fellows and experts in relevant fields to participate on panels about such issues as planning for the fellowship and tax and financial concerns. Workshops on grantwriting and career development would help position fellows after their fellowship.
In response, PFA plans to establish a peer-to-peer mentor program that would connect new fellows with previous ones. PFA also plans to investigate potential partnerships with local institutions to provide the fellows technical assistance.
Assess Communications Strategy
PFA's most important constituency are local artists and Philadelphia arts leaders and funders, and so materials and dissemination tactics should be designed with these stakeholders' information needs in mind. For example, it is important that local artists be aware when their respective disciplines are the categories up for consideration. The aforementioned newsletter, which is distributed to the entire PFA mailing list of 2,800 individuals and organizations, provides ongoing opportunities to restate the processes and objectives of PFA.
In addition, PFA is re-examining its overall communications strategy to ensure that clear messages about the program and its application process are effectively reaching its target audiences.
Gather More Information
A more robust array of data will help PFA more fully understand how its own work benefits individual artists and the community--and will help the Trusts collect facts on PFA's progress. Currently, new fellows complete an "entrance" questionnaire; it could be fleshed out to bring in more data about the artists' lives and work. By adding an "exit" questionnaire and regular follow-up interviews for each cohort--say, every five years--a much more detailed picture of the immediate and long-term consequences of the PFA would emerge.
Working with Planning and Evaluation, the Culture program and PFA are in the process of designing a tracking plan and developing a feasible, cost-effective means of regularly collecting data through these mechanisms.
Janet Kroll and Nichole Rowles are officers in the Planning and Evaluation department at the Trusts.
