Option 1: Incorporate Contextual Information to Make Graduation Rates More Meaningful
There is an abundance of data documenting that the characteristics of beginning students are strongly correlated with their likelihood of graduating from college within a six-year time frame or graduating at all. Put simply, the data document that student success in college depend largely on (1) academic preparation and college readiness² and (2) various aspects of socioeconomic status. ³ Here it is important to note that the characteristics of entering students vary from institution to institution, with the most selective institutions enrolling fewer “at risk” students and open admissions institutions enrolling the most. To derive the most meaning from graduation rate measures within the GRS framework, it is possible to use statistical methods that separate institutional and non-institutional factors impacting student success. This will allow campus and system leaders and policymakers to focus more directly on how well institutions are doing with the mix of students they enroll.

While there are a number of ways to develop such a model, three approaches have gained some credence to date:
1. Actual-to-expected graduation rate model. This model was developed by the Higher Education Research Institute at the University of California at Los Angeles, using data from the Cooperative Institutional Research Program (CIRP) Freshman Survey. By studying degree completion data from 262 participating institutions, researchers were able to identify those factors that distinguished completers from dropouts, such as high school grade point average and parental education. By weighting these factors according to how much impact they had and averaging the estimates for all students at a given institution, they were able to calculate an “expected” completion rate for each institution. A comparison of the “actual” to the “expected” rate provides an indicator of institutional performance.

The Institute found that about two-thirds of the variation in institutional degree completion rates was due to differences in beginning student characteristics. That is, most institutions have an actual rate that is close to the expected rate, but this is not always the case. To illustrate, researchers noted that a public university and a private liberal arts college both have actual completion rates of about 55 percent, and a simple GRS-type presentation would suggest that they are equally effective in graduating their students. When characteristics of beginning students are considered, a different picture emerges. Research suggests that the liberal arts college would be expected to graduate 68 percent of its freshmen while the public university would be expected to graduate only 40 percent. According to this model, the public university is performing better, given its enrollment profile, than the more selective college.

2. Actual-to-peer graduation rate model. The Education Trust developed this model in 2004, the first year that GRS data were released to the public. Researchers conducted an analysis of graduation rates and looked at institutional factors that might explain the wide variation in graduation rates that they found. By performing statistical calculations that take into account some of these factors—including SAT/ACT scores, institutional mission, financial resources, and others—analysts were able to assess how well specific institutions were doing relative to peer institutions that enroll similar students. Some institutions were identified as high performers overall, high performers in terms of effectively serving minority students, and high performers in terms of having made rapid gains over a five-year period. [see Table 1]

3. Disaggregated graduation rate approach. This approach calls for the development of a series of graduation rates for each institution, where the overall rate is disaggregated into separate rates for categories of students known to graduate at different rates. For example, it is known that socioeconomic status correlates with expectation of college graduation and that institutions vary as to the socioeconomic make-up of their enrolled students. Though precise indicators are not readily available, federal student aid eligibility could be used as a proxy measure for socioeconomic status, and graduation rates could be calculated for four subsets of students: those with full Pell Grant eligibility, partial Pell Grant eligibility, subsidized loan eligibility, and no eligibility for Pell Grants or subsidized loans. These disaggregated rates could be compared across institutions rather than a single graduation rate for institutions that vary tremendously in their student bodies.

Option 2: Move Beyond the GRS Framework to Better Reflect Contemporary Student Behavior
All of the options above retain the basic GRS framework of examining only first-time, full-time students, limiting study to a six-year time frame, and focusing only on graduation (as opposed to transfer or continued enrollment). While these approaches greatly improve upon the current simplistic approach, they still cannot capture the full range of contemporary student behavior, particularly at less selective institutions. At these colleges and universities, an increasingly large proportion of students enroll part-time, persist for more than six years, and/or enroll in multiple institutions on their way to successful graduation. If institutional leaders and policymakers want a more comprehensive understanding of the full dimensions of institutional performance, they need accountability measures that correct the shortcomings of GRS. New tools are needed to analyze and communicate a wider range of student outcomes and new means to capture them.

1. New methodologies. In the mid-1990s, AASCU, the American Association of Community Colleges, and the National Association of State Universities and Land-Grant Colleges sponsored the Joint Commission on Accountability Reporting (JCAR) that proposed a new template for accountability reporting. Specifically taking into account the fact that many students attend part-time, “stop out,” transfer, and take longer to graduate, JCAR developed a comprehensive methodology for measuring student advancement designed to promote accurate comparisons among institutions. In 1997, the U.S. Department of Education authorized the use of JCAR conventions as an acceptable form of Student Right to Know Act compliance.

JCAR went beyond GRS in several dimensions. First, while GRS considers the simple graduation rate as the only indicator of student success, JCAR considers that students are successfully advancing if (1) they have graduated, (2) they have transferred, or (3) they are still enrolled at the institution. The rationale is that all of these outcomes, in contrast to non-enrollment, represent positive steps toward degree attainment rather than negative commentary on the institution. Second, while GRS looks at a single point of time—six years for a baccalaureate degree—JCAR calls for measures to be taken at three points in time: (1) catalog award time (four years for a bachelor’s degree), (2) extended award time (six years for a baccalaureate degree), and (3) eventual award time (allowing part-time and discontinuous enrollments more time to complete). Third, while GRS looks at first-time, full-time freshmen only, JCAR includes all students new to the institution in a given fall term, including part-time and transfer students. It does recommend that separate measures be reported for all first-time students (including part-time), transfer students, and the standard GRS cohort of first-time, full-time freshmen only. [see Figure 1]

2. New means. In order to implement a JCAR or similar model that goes beyond the current scope of GRS, comprehensive data systems are needed that cross institution, system, and state boundaries and that track students for longer periods of time. Much progress has been made in this direction, but much more needs to be done.

Specifically as a result of the Student Right to Know Act and GRS, many state higher education agencies and system offices have made tremendous investments in student unit record data systems over the past 15 years. These data systems contain student-level data from multiple campuses and terms, and can link individual student progress through a unique identifier such as a Social Security number. Many unit record data systems currently track students across institutions in a particular state or system, and have the ability to capture transfer behavior among public institutions within the state. Comprehensiveness and capabilities vary widely across states and tracking students across state borders, as well as to private institutions, are the exception rather than the rule.

To the extent that state and system databases can talk to one another, it is possible that in the future, these databases could be linked into regional or national networks that would greatly enhance analytical capabilities. In 2003, the Lumina Foundation for Education conducted a study of state- and system-level unit record databases to determine the feasibility of linking existing databases to achieve better student progression data. The study concluded that there are obstacles to developing such a network, but they are not insurmountable and the resulting network would provide a much more comprehensive picture of student progress. [see Figure 2]