Return to Assessing International Learning

Assessment Methods

Selection Process

FIPSE-funded project institutions used a variety of criteria to review more than 20 assessment instruments already in use to determine which instruments and approaches could be adapted to assess the identified learning outcomes. These criteria both reflected the goals of the project and acknowledged good practice in assessment. The methods reviewed included: knowledge tests, portfolios, interviews, oral proficiency examinations, satisfaction surveys, and cross-cultural inventories. After careful consideration of the strengths and weaknesses of various types of assessment (52KB; PDF), the working group narrowed the choices to an electronic portfolio (ePortfolio); the Beliefs, Events, and Values Inventory (BEVI); and the Intercultural Development Inventory (IDI).  As the project evolved, the decision was made to focus project attention solely on the ePortfolio methodology.

Criteria for Selection

The group’s foremost consideration was to focus clearly on student learning. There was consensus that any assessment approach must provide direct evidence of students’ knowledge, skills, and dispositions, and adequately address deep, complex learning rather than merely tapping surface learning. In other words, the choice of assessment strategy was driven by and aligned with the nine previously defined international learning outcomes. In this way, the group avoided a common mistake in assessment, namely allowing their outcomes to be defined de facto by the available instruments. Another consideration was comprehensiveness: The approach adopted needed to be broad enough to address all or most of the learning outcomes, and do so in an integrated and developmental way that would provide a nuanced picture of student learning for purposes of improvement as well as documentation and celebration. The ePortfolio, as a collection of different kinds of student performances and products, seemed ideally suited to accomplish all this.

The project also raised technical and practical concerns. The working group was committed to developing an assessment strategy that would offer validity and reliability. That is, the assessments actually needed to assess what they purported to assess, and they needed to do so consistently, both within and across institutions. At the same time, it was essential that any strategy be adaptable enough to work for different samples of students, at different capture points, in different programs, across the full range of participating institutions.

The practical concerns included cost, on-campus acceptance, sustainability, and replicability. Costs of the project, including both monetary and opportunity costs, needed to cover development and implementation of the project (including developing, adopting, or adapting instruments; training participating faculty and administration; data analysis and interpretation; and use of results), but also stay within the means of participating institutions. Because assessment of student learning can be a contentious issue for faculty, particularly in the case of little-explored terrain such as international learning, the working group also was sensitive to the question of acceptance. The portfolio, which is already widely used and recognized for its ability to yield helpful information, enjoys a high level of acceptance, particularly among liberal arts faculty, and thus seemed a prudent choice from that point of view.

A fundamental expectation of the project, of course, is that the strategy will receive appropriate support, become institutionalized on participating campuses, and over time be used routinely to gain insight and effect ongoing improvement. Only then can the benefits of assessment be fully realized. If this is the expectation, however, then sustainability must be built in from the beginning. What this means specifically will differ on each campus according to its mission, size, student body, and campus culture. In general, however, sustainability requires selectivity, pacing, demonstration of the benefits of the assessment process, campus-wide communication, and visible rewards.

Electronic Portfolio (ePortfolio)

The only instrument flexible enough to address all of the selected outcomes was the electronic portfolio (ePortfolio). To ensure its effectiveness, the working group developed a set of performance indicators and rubrics to measure each of the learning outcomes. Students in the project are asked to include evidence of having achieved all of the agreed-upon goals in their ePortfolios. The foreign language communication goal requires the participation of faculty members with communicative competency in relevant foreign languages in the rating process. While the group has identified sample instruments—such as the Oral Proficiency Inventory—for this purpose, each team has the option to select an appropriate tool for determining communication competency.

Each project institution also has the option to use an additional assessment method, such as an inventory (e.g. the Intercultural Development Inventory (IDI); or the Beliefs, Events and Values Inventory (BEVI).  The use of an additional quantitative assessment instrument may provide indirect evidence of changes in student attitudes, or additional insight into values, affective development, and students’ potential for growth.

Please direct questions about this page to:
jill_wisniewski@ace.nche.edu |  Staff Contacts
Download our  information brochure
This page last updated on: 08/07/2007