Strengths:
This approach offers a direct measure of college-level learning, since it occurs throughout the undergraduate experience. It serves multiple stakeholders and purposes, including accountability, state policy development, and institutional improvement.

It takes into account differences in student input and lends itself to the development of models and benchmarks for diverse institutional types.

Because it is explicit about collegiate-level skills, it facilitates academic alignment with the K-12 sector.

Challenges/Limitations:
Historically, there has been a great deal of autonomy concerning assessment of student learning at the classroom, department, and institutional levels. Although learning assessment is increasingly a part of the accreditation process, many faculty and administrators remain resistant to state-level “interference” in academic matters such as assessment of learning.

More research is needed on how to develop benchmarks and models for different institutional types.

Unlike the indirect approach, the direct approach does not point to specific directions for institutional improvement.

Unlike the applied approach, the direct approach does not measure how college learning relates to real-world performance. It cannot capture the full impact of the college experience that continues to unfold as graduates gain maturity and experience.

Strengths:
This method provides a useful proxy for direct learning assessment.

It yields useful information about specific institutional strengths and weaknesses and lends itself especially well to institutional improvement efforts. Through Project DEEP (Documenting Effective Educational Practice), the NSSE Institute for Effective Educational Practice has examined the workings of 20 successful institutions and is sharing findings in order to help institutions identify strategies for using NSSE data to increase student success.

The approach is useful for diverse institutional types and peer information is available.

Challenges/Limitations:
Indirect assessment does not—and never will—measure actual student learning. In order to identify specific learning strengths and gaps, other types of data would be needed.

There are always questions about the reliability of self-reports. NSSE developers have addressed this issue by identifying five conditions under which self-reports are valid:

(1) when the information is known to the respondent;
(2) when the questions are clear and unambiguous;
(3) when the questions refer to recent activities;
(4) when respondents think the questions merit a serious and thoughtful response; and
(5) when answers do not threaten, embarrass, or violate the privacy of respondents.

Strengths:
This method measures outcomes several years out of college, when institutional effects have had time to more fully develop. It reflects implications for the real world, as assessed by employers and alumni in the workforce.

There is potential for development of benchmarks and models for different institutional types, based on existing data.

Challenges/Limitations:
Given the passage of time and intervention of other factors, it may be difficult to tease out institutional effects on students.

There are questions about the reliability of self-reports; given the passage of time, alumni may not be able to report accurately about college experiences.

a. Institutional improvement vs. public accountability
A value-added assessment program can—and arguably should—satisfy both objectives, but the relative emphasis will vary across states and systems. For example, states or systems wishing to focus on campus learning environments may concentrate more on indirect measures, while those more concerned with workforce readiness may give more weight to applied metrics. The point is that there should be a “fit” between the mix of approaches selected and the policy priorities of a particular state or system.

b. Relationship to other components of the educational pipeline
Value-added learning assessment cannot exist in a vacuum, and thus, must be structured so that it complements other quality assurance mechanisms in the educational pipeline. How would a value-added program relate to high school exit or college admissions and placement exams? In many states, such questions call for the engagement of P–16 entities (provided they are active and influential) in developing, testing, and implementing a comprehensive value-added system.

c. Linkages across higher education policy
The vitality and success of a value-added program also will depend on its connection to key areas of policy, including:

Accountability—What weight should be given to value-added metrics, particularly in relation to existing outcome measures (e.g. persistence, graduation, and post-baccalaureate placement rates)?

Finance—Will (or should) the data gleaned from such a program play a role in funding allocation (either base or supplemental)? If so, to what extent?

Access and Inclusion—In what ways can value-added assessment data be used to assess and recalibrate state and system policies designed to promote college participation and success for historically underrepresented and disadvantaged groups?

Economic/Workforce Development—How can assessment findings be used to create a feedback loop with state economic development organizations and the private sector regarding the fit between what colleges and universities are producing and what the state needs or will need?

a. Resources
The experiences of the National Forum on College-Level Learning and similar initiatives reinforce that while effective learning assessment requires significant and sustained financial investment, securing consistent policymaker support is even more critical—and often difficult. States and systems should think broadly in terms of securing needed leadership and logistical support. This may include the reallocation of existing resources from obsolete or lower priority accountability functions.

b. Participation
As with any new initiative, building a critical mass of interest and substantive involvement can be a challenge. What level of student/stakeholder participation is necessary for the selected assessment measures to be credible? How can that level of participation be garnered and maintained? How can the quality of stakeholder participation be assured, particularly if it is voluntary or not linked to academic advancement? Can sufficient statistical samples be developed and maintained across different groups (e.g. first generation, low income, racial/ethnic minorities) to accurately gauge differences in perception and performance? The National Forum on College-Level Learning has highlighted this as a key issue and states and systems must develop a participation strategy as part of its implementation process.

c. Application
How the value-added program is applied across a wide range of campuses in a system will greatly affect its utility and relevance. For example, will the program establish goals or benchmarks for institutions, either for individual measures or for a composite of measures? Will those goals/benchmarks account for differences in institutional mission and admissions selectivity? How will the resulting data be presented and communicated, particularly to ensure that they are understood by and useful to a broad array of internal and external stakeholders?