ACE | Toolkit - Rubrics

Guide to Assessing International Learning

•	Preparing for Assessment
•	Overview of the SPIF/ePortfolio Approach
•	Implementing the SPIF/ePortfolio Approach
•	Using the Tools
•	Using the Results
•	Other Assessment Resources
•	Lessons Learned

Lessons Learned

The raters involved in the ACE/FIPSE project provided feedback to the assessment coordinators about the original ACE/FIPSE rubric. While the raters were generally pleased with the quality and user-friendly format of the rubrics, there were two reoccurring issues:

•	Length and complexity: The rubrics were too long with too many decisions that needed to be made.
•	A "not evident" option: Rather than being obliged to mark "inadequate" or make inferences, the raters preferred a "not evident" choice. This option is particularly important if students are not provided with the rubrics to guide them in selecting artifacts to be included in the portfolio.

Several members of the ACE/FIPSE project were concerned that the complexity and length of the rubrics would make it difficult to sustain the SPIF/ePortfolio assessment approach after the project's completion. Consequently, a streamlined version of the rubrics was developed. This version, located on the main page to the left, still addresses the nine originally proposed outcomes, but reorders and eliminates several of the criteria.

Access the original rubric developed and piloted by the six institutions in the ACE/FIPSE project.

Access an annotated version of the original rubrics with explanations of the proposed revisions.

Rubrics are scoring guides that faculty and staff can use to evaluate students. Rubrics developed for assessing specified international learning outcomes are a critical element in implementing an ePortfolio assessment approach. As explained in A Handbook for Advancing Comprehensive Internationalization, "The concept of rubrics is not new; faculty members have always used some form of rating scales and criteria, sometimes implicit, to evaluate student work. Rubrics provide a way of making grading criteria explicit for both grader and student. Good educational practice suggests that the more explicit the grading criteria are, the more students will understand the faculty members expectations, and the more grades will be unbiased and consistent."

Types of Rubrics

Checklist: A simple rubric that indicates whether certain components are present. A checklist does not indicate student competence.

Rating Scales: Often in the form of a matrix, with outcomes on one axis and a scale on the other axis, rating scales show the degree to which certain objectives are present. The rating scale is often on a 1 to 5 scale, or ranges from "poor" to "excellent". These scales, while more precise than a checklist, are often vague and open to subjective interpretation by the grader.

Descriptive Rubrics: Also frequently organized as a matrix, descriptive boxes within the grid contain information about the performance exhibited by a student at a certain level. It allows for very clear and objective communication of expected performance. These rubrics provide detailed information about the objective components as well as the level of performance.

ACE/FIPSE Rating scale

The ACE/FIPSE team created a rating scale to assess the nine learning outcomes agreed upon for the project. In order to provide a rich and textured information for these learning outcomes, each of the nine outcomes was sub-divided further and characterized by individual criteria. Evaluators are asked to rate the students' demonstration of each criterion by using a 1–4 scale, with one being "inadequate" and 4 being "extensive."

In response to rater's experiences using the rubrics and to maximize reliability, the rubrics have been modified. This version is the one we recommend that institutions adapt.

Anchor Portfolios

Anchor portfolios provide important information to raters about what is considered a low, medium, and high level portfolio. The purpose is to help raters be consistent in their evaluations and to promote inter-rater reliability. Below are three anchor artifacts that have been adapted from actual student work. The anchor papers are annotated to reflect how an experienced rater used the rubrics to evaluate student work. These tools are offered as models to be adapted to reflect institution specific outcomes. Such anchor portfolios are especially important for consistency in training rater and therefore maximizing inter-rater reliability in assessing student ePortfolios. See the Rating Process for more information on training raters.

Please direct questions about this page to:
international@ace.nche.edu
This page last updated on: 09/03/2008