Implement & Evaluate Phase

Rothwell, W. J., Benscoter, B., King, M., & King, S. B. (2016). Mastering the instructional design process: a systematic approach (5th ed.). Hoboken, NJ: Wiley.

Formative evaluation is conducted in order evaluate materials and instruction before it is released for mass use. The main goal of this method is to revise and improve the product in order to yield higher results. When developing an evaluation plan there are seven steps that are followed.

The steps involves no only the ID but also leaders, key personnel, supervisors, and so forth. While development the evaluation plan ensuring that required results are meets is very important.

What results are needed in order to improve and revise.

There are five key factors to proper protocol:Decision makers experience with formative evaluationLabelsTimingParticipationMethod of evaluationIf a decision maker is not experienced with formative evaluations ID's must make sure to set the groundwork. The main goal of formative evaluations is to improve materials and instruction.

In this step, the target population is identified for what learners will be involved in the instruction that will be conducted. If the instructional material is for a broad audience then sampling the material with top performers, industry veterans, and notice students would be very important to capture.

What other factors would be important to consider that affect the out come of success in regards to the use of materials, instruction, and student success.

Sets the process of how evaluatons will be conducted.

The Management Plan has all the details such as schedule, events, and task. It also defines the people involved with the formative evaluation, how will results be handled, and who will review the results.

There are four approaches to conducting formative evaluations. The approaches below can be used singular or in combination:

By haveing SME's involved with the design of materials and instruction can be a big benefit for the ID team by providing the most current information that is accurate. There are usually two types of experts for this type of review, some will focus on content and others delivery of content.

This approach gets management and key stakeholders involved with how the instruction will be conducted for the learners that have been identified. This will prove the information on what the students will learn to improve their knowledge or skill, the end result is management and stakeholders will know what their employees were trained on in order to hold them accountable.

In summative evaluations, results are collected after the instruction has been presented to students. These are the results that let the Instructional Designer and other key personnel know what went right and wrong, what needs to improve and how will the deployment of material and instruction change.

The evaluation report is a written document that has defined elements throughout the document such as:Title pageExecutive summaryIntroductionFindings and resultsRecommendations and next stepsAppendixThis documented can be formatted to be converted into other formats such as PowerPoint. Once the report is complete it is then sent out to all key stakeholders for review.

Rothwell, W. J., Benscoter, B., King, M., & King, S. B. (2016). Mastering the instructional design process: a systematic approach (5th ed.). Hoboken, NJ: Wiley.

Revisions to programs or product can be influenced by a number of sources. As shown in this concept map, evaluation reports play a big role in what needs to change. Also keep in mind that other factors can effect change such as management changes and organizational policy changes. Product change can happen based on new information or major software changes that delivery instruction. Any new changes need to be carefully considered and reviewed.

Any significant changes to a product or program will require stakeholder buy off. Usually, if the instruction designed for a private organization requires additional changes after final product completion will be described in the contract of what is allowed and what requires additional review by key organizational stakeholders. The is can be especially true for products that use LMS software due to the expense required to use such products.

Once stakeholders have approved the changes the required revisions will take place. When the revisions are complete the new changes need to be evaluated and presented back to the stakeholders for final approval.

Final product is ready for deployment, which requires additional steps and method to ensure proper delivery of instructions and materials.Rothwell, W. J., Benscoter, B., King, M., & King, S. B. (2016). Mastering the instructional design process: a systematic approach (5th ed.). Hoboken, NJ: Wiley.

A plan must contain a vision for change, such as what will need to change in the future. I can personally relate to this for some course I have monitored, additional areas or discussion would be implemented in course instructions to point out changes and how they will affect a future product. Also part of this vision for change is to ensure alignment of performance goals and making sure that stakeholders are involved.

When instructions and materials are ready for deployment a plan must be in place to do so. Once performance goals are aligned the implementation plan will include how the deployment of self-paced, in-person, and technology-enabled products and materials will be implemented. This can include many other resources such as how the implementation will be handled, will teams or individuals be responsible for certain aspects, when monitoring implementation how will issues be taken care of regardless if a product or student issues arise. Will a master trainer need to be involved in order to prepare multiple facilitators.

Once instruction takes place what other actions are needed to support and get deep roots to stick with what is taking place. If a new program is designed to train managers to be better leaders getting the entire organization to support the efforts will support the diffusion of the intervention.

Once the new knowledge or skill is obtained by the learners will the end result create addtional positive results for the organization that requested the change. Will the students that return back to their normal organizational role close performance caps that created the need for instruction in the first place? If so the ogranization will adopt the training as a solid product.

If the new instruction gains support and approval of the entire organization will the instruction become compliance or comment to using the new program or product for future? These two approaches come with different results and requirements for the organization involved.

Monitoring must be used in order to determine what changes need to be in order to create better success and know that performance goals are being reached. Monitoring can create additional requirements such as who needs to know, performance reports, and how is it furthering organization success.

Once new changes are discovered the Instructional Designer needs to make sure if they should take place soon or later. Also, time and cost need to be known for any new changes. Changes that are known need to create a positive outcome, just because changes are required does not mean action needs to be taking before proper planning of how it will be implemented.

According to Branch there are three common componenets of an Evaluation Plan:Summary outlining the purpose, data collection tools, timing and person/group responsible for the level of evaluation.Set of summative evaluation criteria.Set of evaluation tools.Branch, R. M. (2010). Instructional Design: The ADDIE Approach. Boston, MA: Springer US.

Evaluations are broken into three levels which all are guided by the "5 w's" Who, What. When, Where, Why, and How.

This level measures the comforts that many students like. For example course content, classrooms, and resources available. Many courses that I have attended or oversaw measured student perception through the use of end of course critique and open discussions.

This level measures if students can complete the task that was specified in goals and objectives. There are many methods of measurement such as test and observations.

Here students knowledge and skills through performance are evaluated, such as when the student returns back to there assigned job duties. Some methods to determine if the instruction has been effective is through work performed, supervisor feedback, and observations.

Tools are selected to determine if performance gaps were closed. These tools are selected before the instructional material is presented to students.

Evaluations are carried out by other people that will have a constant role in administering the evaluations after each instructional course. Evaluation methods are set and the evaluation plan is made. This plan contains all three levels of evaluations, containing the "5 w's."

Branch, R. M. (2010). Instructional Design: The ADDIE Approach. Boston, MA: Springer US.

At this point of the implementation phase deciding the teacher that is need to instruct the course is known. Comparing applicates qualifications and skills will be under review.

Identifying a qualified teacher is now underway. Depending on the course to be facilitated some teachers might need to be certified in the field they will be instructing.

Once an instructor is decided depending on the material or skill taught scheduling them for a train the train might be appropriate.

Instructors are notified of the Train the Trainer schedule and course information. Additionally, this type of course prepares the teacher not only for the material that will be presented to students but will also close any learning caps or unknown knowledge that the teacher did not have. There are various methods to how a Train the Trainer will be conducted. From personal experience, the teacher can be the student, but also switch roles during this type of course in order to gain insight of both roles.

All learning resources have been prepared, knowledge and skills that are need for students to successfully complete the instruction is set.

Students have been determined through prerequisites, knowledge, and skills.

The schedule will indicate the number of students, where the instruction will take place, and so forth.

Students are identified and they receive course information and what they should expect.