ACE | Web Guide - Using the Results

Guide to Assessing International Learning

•	Preparing for Assessment
•	Overview of the SPIF/ePortfolio Approach
•	Implementing the SPIF/ePortfolio Approach
•	Using the Tools
•	Using the Results
•	Other Assessment Resources
•	Lessons Learned

A cautionary note...

As the assessment results start coming in, the team and other stakeholders may be inclined, in their enthusiasm about the assessment findings, to use the results in ways that were not initially envisioned. One of the most important principles of good practice in assessment is being transparent with faculty and students concerning the intended usage of the assessment results. While assessment results may seem gloriously useful, for example, to advocate for additional funding, it is best that the assessment team be cautious in how the results are used. The same logic may raise concerns for some faculty and administrators about results being used in an adverse fashion—such as determining which programs should be cut, etc. Once again, being clear and consistent about your purposes for engaging in assessment are critical from the start and throughout the process. For more information, see Developing an Assessment Plan and Lessons Learned.

Prioritize or refine global learning outcomes

Through the assessment process, institutional teams may find that the student global learning outcomes that were articulated are not achievable by students on the scale and scope originally envisioned. Teams may elect to revisit, for example, whether they should expect all graduating students to achieve the articulated outcomes.

Another issue that is likely to emerge is whether students have had sufficient opportunities to achieve all of the articulated outcomes. It may be more realistic to select a few of the outcomes that are most appropriately aligned with the mission of the institution, or a particular college/school, academic/administrative units or programs.

Through the assessment process, institutions may discover they need to adjust their expectations concerning how many outcomes they can realistically assess during a given assessment cycle. If the team finds that they have much more data than they can use, this indicates that for the next round, they could focus on a few outcomes rather than the full range developed. This was one of the lessons of the FIPSE project. Attempting to assess nine learning outcomes one of which was proficiency in speaking, listening, reading and writing a second language proved to be very ambitious.

Refine the assessment process and instruments

When reviewing the results from the first round of assessment, it is important to think of this round as a pilot of the assessment process and instruments. While surprising results should serve to foster important conversations about international learning (as noted below), they should also serve as a flag for your team about potential adjustments needed in the assessment process. The following process issues may be worth considering:

The instruments developed through the ACE/FIPSE project were developed with the intention of raising the broadest array of questions about student learning and addressing the broadest array of global learning opportunities that institutions may provide. Consequently, the instruments were designed to be adapted by institutions as they used them. After piloting the instruments, your team should:

Stimulate reflection and propose improvements in international learning experiences

Every stage of the SPIF/ePortfolio assessment approach can catalyze important conversations about student's international learning. If designed thoughtfully, the creation of the ePortfolios can enhance student reflection about connections between their international learning experiences. When engaged in the rater training experience, faculty may realize opportunities for revising courses they teach to more effectively assist students in achieving the specified learning outcomes. In sum, the SPIF/ePortfolio assessment process affords formative opportunities for improvement of student learning.

When the responses from the SPIFs and the ratings from the ePortfolios are brought together, multiple analyses are possible (see strategies for data analysis). The data tables and charts generated from such analyses can then be shared with the faculty and other appropriate stakeholders to discuss what these results say about the state of international learning at the institution. Hypotheses could be crafted that can then be tested out through another round of assessment, and in the interim, through additional conversations with all involved. Surprising findings have the most potential to stimulate faculty and others to revisit assumptions about their students or course and program offerings.

Ultimately, the international learning outcomes and assessment process is intended to assist institutions in making data-informed improvements in their international learning experiences. If after an additional round of assessment, the hypotheses that were developed initially are confirmed, then there is sound evidence to suggest appropriate revision of programs and courses.

Several of the institutions in the ACE/FIPSE project gleaned important information through the implementation of the SPIF/ePortfolio approach. One institution, Portland State University, complied sufficient comparative data to engage their colleagues in meaningful discussions about the findings. An overview of some of their findings is available.

ACE/FIPSE Institutional Plans for Continued Assessment of International Learning

All of the institutions in the ACE/FIPSE project are committed to continuing their effort to assess international learning and to implement at least some aspects of the SPIF/ePortfolio approach. The following plans highlight their intentions: