Observation Calibration Training
It Works!
North Carolina Improves Calibration of their Evaluators with Observation Engine
The North Carolina Department of Public Instruction (NCDPI) is using Observation Engine™ to improve the calibration of teacher evaluators. And it works! In the pilot of the Observation Calibration Training (OCT) during the 2014–15 school year, NCDPI worked with administrators from 20 LEAs. The analysis showed that there was statistically significant scoring improvement from the first calibration assessment to the second. In addition, the improvement in scoring was directly related to the number of exercises each observer completed-–the more Lessons observers completed, the more their scoring improved.
About Observation Engine
Observation Engine is an online tool that uses classroom videos to ensure accurate and reliable classroom observations. With this tool, North Carolina educators can access observer calibration events & reporting that provides practice with rating and helps increase rater agreement to target scores. The online platform allows participants to watch classroom videos, observe instruction, and rate teachers using the North Carolina Educator Evaluation System rubric that is used to evaluate all teachers in North Carolina. Participants receive immediate feedback as to the suggested ratings and supporting evidence for that particular rating. Observation Engine will promote continuous NCEES improvements through an easy to use platform.
The complete process includes:
· Scoring Study at the beginning of the OCT that serves as a pre-test
· Scoring Study at the end of the OCT to determine growth
· Lessons within the OCT include:
-2 full video lessons lasting an entire class period
-17 mini lessons that are element specific
The element lessons are the 17 observable lessons identified within the N.C. Professional Teacher Standards.
During the 2015–16 school year, NCDPI is rolling out the OCT to administrators across the state. OCT is structured to be individual and self-paced. However, survey data from the pilot suggests that participants benefited from the process when it was facilitated by a leader in their LEA/Charter. There will be a recommended Facilitators Guide for leaders to use if they decide to participate as a group. This implementation will focus on establishing a collaborative element to the experience, encouraging administrators to engage in conversations around the OCT process.
The OCT will start in November and continue through the end of school year (2015-16). If you are interested in participating in the 2015-16 Observation Calibration Training. Additional information will follow.
OCT Webinars
Observation Calibration Training (OCT) Introduction and Overview Webinar
Intended audience: OCT Participants
Please register for Observation Calibration Training (OCT) Introduction and Overview on Dec 10, 2015 4:00 PM EST at:
https://attendee.gotowebinar.com/register/80793584091348482
The North Carolina Department of Public Instruction is committed to providing high quality professional development to school leaders to support their ability to offer effective guidance to teachers. To address this goal, the department is offering Observation Calibration Training (OCT) for school administrators across North Carolina. Observation Engine is an online tool that uses classroom videos as a context for increasing the consistency and accuracy of an evaluator’s ratings. This webinar is the first of a series and will include an introduction and overview of the OCT training. Contact: kimberly.simmons@dpi.nc.gov
After registering, you will receive a confirmation email containing information about joining the webinar.
Thursday, Dec 10, 2015, 04:00 PM
undefined
OCT: Manager Role Webinar
Target Audience:
OCT Participants
Description:
This webinar will walk LEA facilitators through the Manager account role of Observation Engine. Facilitator's will have access to reports and be able to monitor their observers’ progress through the OCT.
After registering, you will receive a confirmation email containing information about joining the webinar.
Registration URL:
https://attendee.gotowebinar.com/register/674817936730033410
Contact:
Kimberly Simmons
Thursday, Jan 21, 2016, 02:00 PM
undefined
OCT: Scoring Study #1 Data and Facilitator Support
Target Audience:
OCT Participants
Description:
Part 1 of this webinar will provide OCT participants with an overview of the Scoring Study #1 results including a calibration discussion highlighting areas of alignment and non-alignment. Participants will learn possible reasons for lack of calibration and concrete strategies for increasing personal accuracy on calibration. A personal plan for prioritizing the OCT lessons in order to increase calibration will be reviewed.
Part 2 will support OCT leads facilitating the OCT. We will identify barriers and successes as we dig deeper into the craft of evaluation and the improvement of inter-rater agreement.
Registration Link:
https://attendee.gotowebinar.com/register/4263554616508740097
Contact:
Kimberly Simmons - kimberly.simmons@dpi.nc.gov
Monday, Feb 8, 2016, 04:00 PM
undefined
OCT: Check-in Webinar
Intended Audience: OCT Participants
https://attendee.gotowebinar.com/register/3607820177828870402
Information will include updates, support information and resources based on participation and performance.
Contact:
Kimberly Simmons - kimberly.simmons@dpi.nc.gov
Tuesday, Mar 1, 2016, 04:00 PM
undefined
OCT Wrap Up Webinar
Target Audience:
OCT Participants
Description:
Please register for OCT Wrap up Webinar on May 19, 2016 4:00 PM EDT at:
https://attendee.gotowebinar.com/register/7333300923780603650
The final webinar will examine areas of improvement in calibration across the entire group of evaluators who participated in the two pilot scoring studies. It will also explore how reports provided by Observation Engine can be used to customize professional development and support collaborative learning opportunities in your LEA.
After registering, you will receive a confirmation email containing information about joining the webinar.
Contact:
Kimberly Simmons
Thursday, May 19, 2016, 04:00 PM
undefined
Have questions about Observation Calibration Training?
If you have questions regarding the OCT, please contact Kimberly Simmons, North Carolina Educator Evaluation System Consultant.
Email: kimberly.simmons@dpi.nc.gov
Website: ncpublicschools.org
Phone: (828)-406-9237