Rating Process
Selecting and Training Raters
External Stakeholders:
Members of the community and alumni
faculty and students may complement your rating teams as ePortfolio
reviewers. This would be particularly useful if your community and
alumni base includes people who speak languages that are included in the
ePortfolios but not accessible by your team.
An effective ePortfolio assessment approach depends upon a team of
trained raters. These raters need to be well versed with the rubrics
that are to be applied when evaluating the artifacts in the ePortfolio.
Those involved in the ACE/FIPSE project found that training sessions
were critical for ensuring consistency in ratings and therefore
maximizing inter-rater reliability.
In selecting raters, the following points should be considered:
- There must be incentives to attract a robust pool of raters. These
incentives can include monetary incentives or recognition within
departments for service to the institution
- Raters can be faculty or graduate assistants. However, all raters
should have experience and knowledge of international learning
In training raters, consider:
- Providing raters with background review materials, including the
Handbook for Raters and anchor ePortfolios, before the training session;
sample
anchor portfolios are available
- Allowing adequate time for training sessions, including time to do
"test" evaluations of ePortfolios
- Conducting wrap-up meetings after each rating session, to revise and
refine the rating process and address issues
Handbook for ePortfolio Raters
This Handbook, created for the ACE/FIPSE project, gives raters an
overview of the SPIF/ePortfolio assessment method and the process of
rating student ePortfolios using rubrics. The Handbook contains
project-specific material, but is intended to be adapted as a training
manual for other assessment efforts. Access
the Handbook here.
Please direct questions about this page to:
international@ace.nche.edu
This page last updated on: 09/03/2008
| international learning outcomes, rating, rubrics, assessment fipse ace |
Related Files
Rater_Training (Word Document)
|
