Assessment of Higher Education Learning Outcomes (AHELO)

Organization for Economic Co-operation and Development (OECD)

A feasibility study is scheduled for implementation in 2010–11. OECD is asking each country/state participating in the feasibility study to contribute €150,000 (approximately $210,000). In the United States, institutions in four states (Connecticut, Massachusetts, Missouri, and Pennsylvania) will be working together and with the State Higher Education Executive Officers (SHEEO) association to participate in the feasibility study.

In addition, OECD has launched a campaign to raise external funds from a range of foundations, organizations with an interest in higher education quality, and the corporate sector (with a focus on large employers of economics and engineering graduates).

Barbara Ischinger, Director of Education; Richard Yelland, Head of the IMHE Program; Andreas Schleicher, Head of IA Division; Karine Tremblay (Project Manager)

The OECD Assessment of Higher Education Learning Outcomes (AHELO) aims to assess the feasibility of capturing learning outcomes on an international scale by creating measures that would be valid for all cultures and languages. OECD hopes that between 10,000 and 30,000 higher education students in more than 14 countries will take part in a feasibility study to determine the bounds of this ambitious project, with an eye to the possible creation of a full-scale AHELO upon its completion. The object of the study is to determine whether an international assessment is scientifically and practically possible. The assessment will be done at an institutional level. It is expected that the assessment will be based on a written test of the competences of students who are almost at the end of a bachelor's degree program.

The AHELO feasibility study will consist of four strands: generic skills; discipline-specific strands in engineering and economics; and a value-added strand. The generic skills strand will be tested using a version of the Collegiate Learning Assessment (CLA), adapted to the international context. Assessments will be developed for engineering and economics. For each assessment, contextual data will be used to obtain information on the context for learning in the following domains:

• Physical and organizational characteristics. Observable characteristics such as enrollment figures or the ratio of male students to female students.
• Education-related behaviors and practices. Student-faculty interaction, academic challenge, emphasis on applied work.
• Psychosocial and cultural attributes. Career expectations of students, parental support, social expectations of higher education institutions.
• Behavioral and attitudinal outcomes. Students' persistence and completion of degrees; continuation into graduate programs or success in finding a job; student satisfaction, improved self-confidence, and self-reported learning gains claimed by students or their instructors.

The contextual data will be gathered from public statistics and previous research, as well as collected through surveys of students, faculty, and HEI administrators. During the long term, OECD hopes to develop alumni surveys and employer feedback reports for future inclusion in a full-scale AHELO.

The value-added strand is the most complex and least developed at this point. OECD states that "this strand of work will... reflect on possible methodologies, drawing upon similar work that has already been carried out by the OECD at the secondary education level. Researchers will consider the merits of existing methodologies, and examine psychometric evidence... on the basis of existing data collected at the national level."

As of March 2010, the following countries have agreed to commit institutions to participate in the following strands of the feasibility study:

• Generic Skills (CLA): Finland, Korea, Kuwait, Mexico, Norway and the United States (four states).
• Engineering: Australia, Japan, Sweden.
• Economics: Belgium (Flemish Community), Italy, Mexico (to be confirmed), the Netherlands and the Russian Federation.

As noted above, OECD is still in the process of recruiting U.S. participants, primarily for the generic skills assessment, and hopes to complete that work by September 2009.

International experts have been consulted in seeking to define the scope of the task. Three ad hoc experts meetings on the desirability, possibility, and feasibility of an AHELO were held between April and October 2007. Summary records of the 2007 meetings have been made public. The main points discussed at the third meeting in Seoul, South Korea, in October 2007 were the design and implementation of a feasibility study.

The ad hoc experts group has since been formalized as the "Group of National Experts," and has met four times since (December 2008, April and November 2009, and March 2010) with several additional sub-group meetings on specific topics from 2008 to 2010.

A progress report was presented to an informal meeting of OECD Education ministers in Tokyo, Japan, in January 2008, and received their informal endorsement.

In addition to the experts group, an AHELO Stakeholders Consultative Group (SCG) was established to widen the circle of those involved in the feasibility study. The SCG is an informal group of organizations with a stake or interest in higher education quality (including the American Council on Education from the United States). This group met in Paris, France, in February 2009 and March 2010.

The next phase of the feasibility study will focus on the development of assessment frameworks and instruments in economics and engineering, and the adaptation of the CLA for generic skills, as well as the small scale trial of these three assessment instruments in cooperation with participating institutions to assess cross-linguistic and cross-cultural validity.

OECD's plan calls for development of the assessment instruments in 2010–11 and implementation of the feasibility study during the 2011–12 academic year.

www.oecd.org/edu/ahelo

03/30/2010