View Portfolio

Transparency and Accountability “College Portrait” Project Wins Early Praise

The new Voluntary System of Accountability (VSA) and the accompanying web-based statistical template known as the College Portrait, created by the two associations representing public four-year higher education, are still being refined, but members of the panels that labored for more than a year to bring the projects to fruition say they’re delighted at the early response.

At press time, nearly 120 public university systems and individual public campuses had signed up for the project. At their annual meetings last November, the boards of directors of both the American Association of State Colleges and Universities (AASCU) and the National Association for State Universities and Land-Grant Colleges (NASULGC) voted unanimously to encourage their members to participate.

The overall goals of the VSA and the College Portrait are to provide more and better information to:

    • Demonstrate accountability and stewardship to the public

    • Measure educational outcomes to identify effective educational practices

    • Assemble information that is accessible, understandable and comparable

The project is, in part, AASCU leaders acknowledge, a response to public and governmental pressure for colleges to provide more evidence of what taxpayers and individual students and their families receive for their investments in higher education. The project also is designed to allow students and families to make more direct comparisons among colleges of such things as the actual costs to a particular student, students’ satisfaction with their college experience, and documented gains in student achievement.

While the recently concluded Spellings Commission is often cited as the impetus for VSA, the real concern lies deeper.  Says George L. Mehaffy, vice president for academic leadership and change at AASCU, “I believe the American people have a deeply-felt anxiety about the nation’s capacity to be competitive in the future, in the face of rising competition from international competitors.” They also display, he says, “a pervasive unease about college costs as they are presented in the media. They worry about whether they will be able to afford college for their children.  For the first time in our nation’s history, parents are beginning to wonder whether their children will be able to have the same standard of living they have enjoyed. These concerns are often expressed as political and policy questions.”

As a result, “There is clearly going to be greater accountability for higher education in some form, and it seems only reasonable to try to steer the movement in a direction that makes sense,” says Mehaffy, whose division is providing leadership for the VSA project at AASCU. “This ultimately is about the higher-education community controlling its own destiny. Either we are in charge or somebody else is, and ‘somebody else’ is never the right answer.”

Dozens of presidents and other officials from AASCU and NASULGC campuses were involved in the process of deciding what data to include in the statistical College Portrait, how data currently available could be improved, and how the data should be presented publicly (for more information, see voluntarysystem.org and Public Purpose, September/October 2007, p. 6).

Much of the information included in the College Portrait, which will go on the websites of all participating institutions, is based on data currently available, with some notable modifications and refinements to provide such elements as a better “cost calculator” for individual students and a much better estimate of an institution’s graduation rate.

In addition, the California State University System (CSU) is encouraging other VSA participants to follow its lead in adding a section to the portrait called the “Contributions to the Public Good,” a page 6 to what was originally a 5-page report. That sixth page on the CSU campuses’ websites includes such items as data on the volume of degrees granted, the economic diversity of students, and statistics showing what the average CSU student debt of its graduates is relative to state and national averages.

The evolution of the VSA and the College Portrait is ongoing, and an oversight committee is in place to guide further developments and refinements. It is headed by William (Brit) Kirwan, chancellor of the University System of Maryland, and Jolene Koester, president of California State University Northridge.

Says President Koester: “One of the really important elements to keep in mind regarding the VSA is that we have a structure by which to improve it. We have a framework now that is valuable but that is mutable and open for refinements by member institutions.” She adds,  “Higher education historically has been very responsive to societal needs and expectations, and the collective higher-education response to the VSA is pretty consistent with that tradition.”

Sally Clausen, president of the University of Louisiana System (ULS)—the first system in the country to sign up for the VSA—sums up her system’s attitude this way: “We know there is much more to learning than what the test results show, yet measurement is important. Graduation and retention rates are important. And the public is interested in knowing more about its public investment.”

Her system’s campuses have been experimenting with different standardized tests of student learning, and the presidents, she says, “have been preparing for something like this for a couple years; they believe it’s a train that’s already left the station.”

The Lumina Foundation provided more than $300,000 to complete development of the current form of the project and to fund the first two years of its operation; previously Lumina awarded AASCU and NASULGC nearly $268,000 for initial development.

Data for Students and Families
To help educate students and families, the College Portrait includes data showing an institution’s “undergraduate profile,” including breakdowns of students by gender, race, ethnicity, age, and geographic distribution. One of the most important elements in this section, supporters say, is the “Undergraduate Success & Progress Rate,” which combines graduation-rate information normally reported to the Education Department with data on enrollment and degrees earned that is collected by the National Student Clearinghouse.  This improved graduation-rate instrument allows tracking of students who transfer in or out of a particular institution, compared to the limited federal graduation rate that only tracks native students who graduate at the same institution. The result, says John Hammang, AASCU’s director of special projects and development, “is a pretty good database beginning about 2000 that is robust enough to tell us who may have enrolled but then transferred and graduated elsewhere.”

Another important element in this section is the “cost calculator,” developed by the University of Texas System, that combines federal financial-aid eligibility with entitlement aid granted by each state or university system. This, says Hammang, “gives a pretty good ballpark estimate” of what aid an individual student might receive and, when combined with the traditional “sticker price,” gives students and families a better sense of what their actual expenses might be.

Richard W. Cost, president of the University of Maine at Fort Kent, who chaired the subcommittee on the cost estimator, said that after looking at a number of different models, “I was somewhat surprised at how many calculators at private colleges were looking pretty good.” Adds Cost, “It is important that we provide that kind of a resource for families. And I hope it opens doors for students who might say they want to go to college, but ‘There’s no way I can afford it.’ Those are the students we really hope to reach.”

Another element of the data provided to students and families reinforces the idea that no “one size fits all” reporting mechanism is appropriate or fair. Participants can select nine areas in which they will highlight for students and families special things about their campuses, from outstanding academic programs to campus-safety systems; from athletics to special international exchanges—whatever they wish to publicize.

Data on Student Experiences and Perceptions
In the section for reporting “student experiences and perceptions,” VSA participants will assess student involvement on campus using one of four existing national surveys that developers believe measure roughly comparable types of student engagement and attitudes. Probably the best known of these surveys is the National Survey of Student Engagement.

Participants will report results for a common set of statements selected for the College Portrait, for example, data involving group-learning experiences and students’ perceptions of their institution’s commitment to help them learn. Participants must have this information on their websites by fall 2009, although some institutions will post it before then.

Data on Student Learning Outcomes
Many participants say that this is the most controversial section of the College Portrait, and research done for the FIPSE will help refine it. Participating institutions can report on specific measures they use to assess students’ learning and intellectual development, including such measures as results on professional licensure examinations.

However, they will also administer and report the results of one of three existing surveys that planners determined would provide comparable types of information on the value that institutions have added to students’ learning. The goal is to show how students have progressed between their freshman and senior years in the areas of critical thinking, analytic reasoning and written communication. The deadline for posting this information is fall 2010.

“This is an attempt to put out information that tells people what happened as a result of going to college,” says Hammang, “although we recognize that one test is not a very good measure of all that happens to a student while in college.  But this new set of measures, however limited it is now as we start this process, marks an important opportunity for the higher-education community to increasingly focus on student-learning outcomes. In the future, our measures will become more sophisticated, more nuanced and more comprehensive. But this is an important beginning.”

Says Richard H. Wells, chancellor of the University of Wisconsin-Oshkosh and one of the developers of the VSA,  “There is no question that the learning-outcomes piece has generated the most concerns from faculty.” Because of ordinary human maturation and a host of other possible intervening influences, he notes, it is hard to attribute specific learning to institutional action. “We do have a lot of work to do on learning outcomes and the types of tools for measuring higher-order learning,” he acknowledges.

Wells has engaged in a wide-ranging effort on his campus to explain and gain support for the VSA among various constituencies. In presenting the VSA to his faculty, Wells notes, “We stressed that it was not meant to replace what we are doing already in outcomes assessment,” including measures of student learning conducted by colleges and departments. This institutional snapshot should complement, not compete with, other outcome assessments already under way or being considered.

To extend the development work of the VSA, the U.S. Department of Education’s Fund for the Improvement of Postsecondary Education has awarded AASCU and NASULGC,  along with the Association of American Colleges and Universities (AAC&U), $2.4 million to further develop the work on assessment of students’ learning.

NASULGC will use its portion of the FIPSE grant to conduct a validation study to better determine whether the three assessment instruments chosen for the VSA are, in fact, measuring the same types of intellectual development, says Hammang.

President Koester of CSU Northridge:  “One of the really innovative parts of this is that the testers have agreed to come together to try to get some uniformity in the psychometric procedures, across tests, so that they all measure the value added by college in the same way. This is an extraordinarily positive consequence of the VSA effort.”

AASCU will use FIPSE funds to measure student growth and skills by looking at behaviors that typically are not measured in the assessment process, Hammang explains. “For example, we’d like to find out if, over time, students develop more sophisticated ideas about what kind of data sources to use. We want to look at undergraduate behaviors that are likely to result in good citizenship, in good workers, in whether students can work as leaders and also as followers, in whether they are aware of the wider world.”

In the third portion of the FIPSE grant, the Association of American Colleges and Universities will focus on the use of student educational portfolios as a better way to assess student learning than standardized tests, says Hammanag. “AAC&U’s task is to develop a national rubric to evaluate these portfolios and to establish a system of cut scores so that we can compare one program to another or one institution to another,” he says. This piece of the research is particularly responsive to faculty concerns, he notes.

Lawmakers and other opinion leaders also are an important audience for the VSA, and California State University officials believe their addition of a “public good” page to the College Portrait is a vehicle to highlight the economic importance of CSU campuses to taxpayers and policymakers, says F. King Alexander, president of California State University Long Beach.

For example, he says, some institutions may have an extremely high graduation rate, yet produce relatively few degree recipients. “If you simply focus on your graduation rate, you can quietly kill your local economy,” he asserts. “You can graduate a very small number of students or you can have a more balanced rate that puts a lot of graduates into such high-demand jobs as teaching, nursing, criminal justice, engineering.” Institutions need to get credit for “building the foundations of our local economies,” Alexander says, noting that he has testified on the VSA both in Washington, D.C. and in California’s state capital and found great interest in the project.

Many presidents either have or are planning to brief local legislators and their staffs on the VSA. Indeed, President Wells of the University of Wisconsin, Oshkosh, says that he has presented the VSA project to legislators and “they love this stuff; they think it is great that we’re willing to do it, so we are engendering a lot more public confidence.”

He acknowledges that when the data are fully available, his campus—along with most campuses—will look good in a lot of areas, but won’t do as well in others. Yet in the end, he says, “I think people will accept that. Legislators understand that we will have gaps, but we are willing to point out that we are trying to get better and that sells.”

Many questions and challenges remain as the VSA and the College Portrait get under way, notes AASCU’s Mehaffy. These include whether the project can eventually somehow be used in conjunction with other systems of accountability currently being developed by private colleges and by for-profit institutions. Another issue is how much students and families actually will use the data in choosing or comparing colleges, he notes, and also whether some lesser-known institutions will actually gain resources from their states if they ultimately show themselves to be more effective than better-known campuses.

Indeed, he says, “I think that if we are successful, it ultimately will be because we have helped institutions develop new tools that they can use for their own improvement.”

No system will be perfect, Oshkosh’s Wells acknowledges, “but if we keep working hard on it, we’re going to have a lot better vehicle for people to match themselves with the academic community that’s best going to serve them.” At the same time, he says, “We really are becoming stewards of place; we’re being respectful and listening more in our communities and connecting more. We’re not going to be able to shape public opinion if we are not going to try to serve it.”

 

Cheryl Fields is a Washington, D.C. writer, editor and communications consultant.