The purpose of the virtual course review is the same as the onsite course review, to evaluate military training courses. Courses to be evaluated are selected by the services, not by ACE. No more than 8 courses can be evaluated during a virtual review. During the virtual review the review team will analyze course materials, identify learning outcomes, and recommend postsecondary educational credit based on its findings. The review is conducted on a secure SharePoint platform. Occupation reviews cannot be done in the virtual environment.
Before the Review
Approximately 90-120 days before a scheduled review (PDF), ACE will alert the service point of contact (POC) about the review and the requirements for submitting course materials.
Approximately 60 working days before the review, Programs of Instruction (POIs), and branch-specific documents such as the Training Course Control Document, Master Course Schedule, and approval letters for courses to be evaluated during the review must be sent electronically to Military Evaluations at email@example.com. For service specific details see Resource section.
ACE must have these course documents on file to answer inquiries from credit-granting institutions regarding course content.
- If the courses are phased, all phases should be submitted for review.
- If there are active duty courses and reserve or other version classifications, they should all be submitted together.
- For courses that were previously evaluated, submit a new POI, if there are any changes other than administrative information (course number, title, training location).
- Courses that have not been reviewed in 10 years should also be submitted for re-review, as recommendations have a shelf-life of only 10 years.
- New courses should be submitted only if they are related to a scheduled review.
- Do not submit courses with fewer than 45 academic hours (except for the CG) unless they are associated with other phases or are a common core course. If you are uncertain please ask.
- Do not submit unit training or correspondence courses that do not have proctored tests.
Credit recommendations from a team review are in effect for 10 years, provided there are no major changes in the course’s content, learning outcomes, or academic hours. Whether a course requires a new team review prior to the 10-year mark is determined by comparing the submitted documents with the most recent course materials that were previously evaluated. If there has not been a substantial course change, ACE staff will code the POI as “CE” for “Credit Extended” and the course will not be re-reviewed.
If there has been a substantial change, ACE staff will code the POI as “PE” for “Pending Evaluation” and will re-review the course.
New courses will be coded as “PE” if they meet the criteria for review.
ACE will prepare a disposition list (PDF) showing all courses submitted and the codes indicating which ones will be reviewed. The disposition list also includes notes indicating any questions the staff has about courses, dates, and previous reviews.
ACE will provide a draft agenda (PDF). The POC should address any scheduling concerns with the ACE field coordinator or the senior program manager.
Virtual reviews are conducted over a one-week period, with the review team working independently and together to evaluate the course materials. Virtual reviews typically begin on a Monday, with an in-briefing by the POC. The team uses an electronic discussion board to collaborate with each other. The virtual review typically concludes with an exit briefing on Friday, which can be conducted via conference call or via email, depending on the service POC’s preference.
Approximately 30 days before the scheduled site visit, the ACE field coordinator will schedule a telephone conference (PDF) to review the course disposition list and agenda and to address any questions or concerns.
The POC should explain to the instructors and course managers the purpose of the review and why current course materials will be required during the review.
The POC should invite anyone responsible for producing course materials to the conference call.
The materials provided must reflect the course content of the POI initially submitted to Military Evaluations for review.
Required materials are:
- A copy of the same curriculum outline (Program of Instruction, Training Course Control Document, and Master Course Schedule) that was sent to ACE with the same training start or implementation dates.
- All associated student manuals and materials, (student guides, exercise books, handouts, etc.).
- All associated instructor materials (PowerPoint presentations, lesson plans, manuals, etc.).
- All assessments. This includes, but is not limited to, examinations, quizzes, go/no-go checklists, evaluation rubrics, and writing assignments.
- Any additional supplemental materials that will enhance the team’s understanding of the course.
Ideally, 100% of the above materials for the course should be provided for the team to review.
However, if the course is too large to provide 100% of the materials, the course manager or POC should provide all assessments and the related instructor and student course content. The course manager or POC should consider the significant modules or content in terms of the learning objectives for the course, the assessments and the related instructional tools (PowerPoint presentations, videos, handouts, worksheets, manuals, etc.).
In situations in which all the materials are not provided, POCs should know that:
- there could be an impact on any credit recommendations,
- during the review, the team may require additional information, and
- the course subject matter expert (SME) needs to be readily available to answer questions and provide additional materials to expedite the review process.
If course materials require a specific security measure (e.g., assessments), please advise the ACE team in advance of the review on how they will be able to review these items.
(Please note that the ACE contract states that classified information shall not be accessed, reviewed, or evaluated for civilian credit equivalency. Any classified content with the course materials must be sanitized before being presented to the team.)
Course materials for virtual reviews are loaded onto a service-specific tab on the ACE SharePoint site. Service points-of-contact must obtain unique passwords from ACE to use the SharePoint site.
All course materials must be loaded on the SharePoint site 21 working days before the start of the virtual review.
For distance learning courses, or courses with online modules, it is preferable to provide the team access to a learning management system to view the files directly. Complex files that require downloading and executing, such as computer based training (CBT) files, do not work well in the SharePoint virtual platform.
The files should be organized and uploaded per the following format:
Electronic File Structure:
Each course should have its own folder labeled with the course number as it appears on the POI submitted to ACE.
The corresponding POI should be the only standalone document in that course folder.
In each primary folder, there should be 3 subfolders for the different types of materials:
In order to ensure efficiency and effectiveness while maintaining academically sound practices, the services should provide documentation that provides clear and concise descriptions of the following components:
Learning objectives. Learning objectives identify the skills and knowledge that must be mastered through successful completion of the learning event. Traditionally, these objectives are defined through terminal and enabling learning objectives and are most often identified in the Programs of Instruction, Training Course Control Documents, and Curriculum Outlines currently submitted by each of the services.
Learning activities. Learning activities describe the method by which the content is delivered and the time and structure of learning events. For resident courses, these activities are often identified in student lesson plans and instructor materials, and supplemented through classrooms, labs, and learning resources (books, journals, equipment, etc .). For distance learning courses, the services should work with the contractor to identify service-specific documentation that provides the review team members with a sense of how the materials are organized and presented; the types of learning resources that are provided; methods of providing learner support; and the manner in which the course developers incorporate course content, its complexity, its rigor, and the level of learning demanded by the learning objectives.
Assessment strategies. Assessment strategies should include identification of the knowledge and behaviors that must be obtained and the indicators that best demonstrate attainment of the intended outcomes. Assessment strategies should also include a description of assessment items, assessment instruments, and assessment protocols. For distance learning courses, the services must demonstrate evidence of a process that ensures firm identification of the student and rigid control of test conditions.
During the Review
Instructor/Course/Subject Matter Expert Availability:
Instructors or course developers need to be "on-call” during the week the team is working on the review. ACE will call or email the POC if the team has questions regarding the course.
In-Briefing (approximately 20–30 minutes):
A general presentation and overview by the POC and any other appropriate personnel helps the team understand how courses being reviewed align with each other and how they fit into the installation’s training mission. The presentation should cover such topics as selection and evaluation of instructional staff, course mission, profile of students attending the course, procedures for evaluation of student performance, and maintenance of student educational records. In addition, any unique nuances of the courses being reviewed should be pointed out to the team at this time.
The in-briefing should also include introductions of key personnel involved in the review.
The ACE field coordinator will briefly describe the background and goals of the American Council on Education and the Center for Education Attainment and Innovation, with particular emphasis on military course reviews.
For a virtual review, the in-briefing is done during a conference call with the review team, the POC, and ACE staff. If the POC is not available, the in-briefing can be done via a PowerPoint presentation that is loaded onto the virtual review portal.
The team's responsibility is to determine whether the course content has the academic subject matter and rigor equivalent to collegiate learning. The team analyzes and evaluates the course materials and aligns the credit recommendation for each course to current college curricula.
This process involves three major tasks:
The formulation of a subject area credit recommendation for each course to include semester hours and credit level.
The preparation of the overall course description, instructional strategies and methods of assessment.
The construction of learning outcomes for each credit recommendation.
When exercising their professional judgment to determine a credit recommendation and whether the course is equivalent to postsecondary level work, team members consider a number of relevant factors, including:
- Course content
- Depth and breadth of material
- Level of difficulty
- Applicability to a range of postsecondary programs
- Learning outcomes
- Evaluative and assessment instruments appropriate to measure the service member’s learning
- Length (usually in contact hours) of instruction for all instructor-led courses
The course review process also incorporates an evaluation of how learning is occurring in the course. This is frequently benchmarked with a learning hierarchy, such as Bloom’s Taxonomy.
As part of the team's process for analyzing course materials, identifying learning outcomes, and making credit recommendations, the faculty reviewers are expected to validate and record the overall passing rate for each course.
As military training typically establishes a course pass rate of 70% or higher, if the pass rate is less than 70%, there is a note printed in the credit recommendation section of the exhibit. If there are graduate-level credit recommendations for a particular course, the pass rate must also be validated and recorded at 80% or higher by the review team.
Using the information provided in the curriculum outline, reviewers craft the course exhibit description which appears in the Military Guide in terms meaningful to civilian educators. The effective date for a course recommendation will be the implementation/training start date of the curriculum outline provided to the review team. If there is an earlier version of the course in the Military Guide, the POI training start date of the current version should align with the end date of the previous version to mitigate gaps in training. If that is not the case, the schoolhouse should also provide the POI and course materials for the interim period for the team to review.
Exit Briefing (approximately 30 minutes):
The exit briefing is an unofficial review report conducted to discuss some initial findings. At that time, the ACE Field Coordinator and reviewers will summarize their reactions to the courses reviewed and provide the unofficial review report of the courses reviewed. It should be noted that the reviewers' job is to determine credit equivalencies; they do not make judgments on how well the school meets its own training requirements or suggest changes, unless formally requested.
In a virtual review, the exit briefing is conducted via telephone conference call. The exit briefing tool, a chart summarizing the team’s review, is emailed to participants prior to the call. Faculty members may participate in the call.
After the Review
- ACE will submit a final report to the point-of-contact no later than 30 working days after the visit.
- If the POC does not raise any issues about the final report, the course exhibits will appear on the ACE Military Guide three days after the report submission to the service.
- If there are questions that require clarification, the results will not be released until they are resolved.
- There is no rebuttal option.