Observation Calibration Training

It Works!

North Carolina Improves Calibration of their Evaluators with Observation Engine

The North Carolina Department of Public Instruction (NCDPI) is using Observation Engine™ to improve the calibration of teacher evaluators. And it works! In the pilot of the Observation Calibration Training (OCT) during the 2014–15 school year, NCDPI worked with administrators from 20 LEAs. The analysis showed that there was statistically significant scoring improvement from the first calibration assessment to the second. In addition, the improvement in scoring was directly related to the number of exercises each observer completed-–the more Lessons observers completed, the more their scoring improved.

Big image

About Observation Engine

Observation Engine is an online tool that uses classroom videos to ensure accurate and reliable classroom observations. With this tool, North Carolina educators can access observer calibration events & reporting that provides practice with rating and helps increase rater agreement to target scores. The online platform allows participants to watch classroom videos, observe instruction, and rate teachers using the North Carolina Educator Evaluation System rubric that is used to evaluate all teachers in North Carolina. Participants receive immediate feedback as to the suggested ratings and supporting evidence for that particular rating. Observation Engine will promote continuous NCEES improvements through an easy to use platform.

The complete process includes:


· Scoring Study at the beginning of the OCT that serves as a pre-test

· Scoring Study at the end of the OCT to determine growth

· Lessons within the OCT include:

-2 full video lessons lasting an entire class period

-17 mini lessons that are element specific


The element lessons are the 17 observable lessons identified within the N.C. Professional Teacher Standards.


During the 2015–16 school year, NCDPI is rolling out the OCT to administrators across the state. OCT is structured to be individual and self-paced. However, survey data from the pilot suggests that participants benefited from the process when it was facilitated by a leader in their LEA/Charter. There will be a recommended Facilitators Guide for leaders to use if they decide to participate as a group. This implementation will focus on establishing a collaborative element to the experience, encouraging administrators to engage in conversations around the OCT process.


The OCT will start in November and continue through the end of school year (2015-16). If you are interested in participating in the 2015-16 Observation Calibration Training. Additional information will follow.

OCT Webinars

Observation Calibration Training (OCT) Introduction and Overview Webinar

Thursday, Dec. 10th 2015 at 4pm

This is an online event.

Intended audience: OCT Participants


Please register for Observation Calibration Training (OCT) Introduction and Overview on Dec 10, 2015 4:00 PM EST at:

https://attendee.gotowebinar.com/register/80793584091348482

The North Carolina Department of Public Instruction is committed to providing high quality professional development to school leaders to support their ability to offer effective guidance to teachers. To address this goal, the department is offering Observation Calibration Training (OCT) for school administrators across North Carolina. Observation Engine is an online tool that uses classroom videos as a context for increasing the consistency and accuracy of an evaluator’s ratings. This webinar is the first of a series and will include an introduction and overview of the OCT training. Contact: kimberly.simmons@dpi.nc.gov

After registering, you will receive a confirmation email containing information about joining the webinar.

OCT: Manager Role Webinar

Thursday, Jan. 21st, 2-3pm

This is an online event.

Target Audience:

OCT Participants


Description:


This webinar will walk LEA facilitators through the Manager account role of Observation Engine. Facilitator's will have access to reports and be able to monitor their observers’ progress through the OCT.


After registering, you will receive a confirmation email containing information about joining the webinar.


Registration URL:

https://attendee.gotowebinar.com/register/674817936730033410

Contact:


Kimberly Simmons

kimberly.simmons@dpi.nc.gov

OCT: Scoring Study #1 Data and Facilitator Support

Monday, Feb. 8th, 4-5pm

This is an online event.

Target Audience:


OCT Participants


Description:


Part 1 of this webinar will provide OCT participants with an overview of the Scoring Study #1 results including a calibration discussion highlighting areas of alignment and non-alignment. Participants will learn possible reasons for lack of calibration and concrete strategies for increasing personal accuracy on calibration. A personal plan for prioritizing the OCT lessons in order to increase calibration will be reviewed.


Part 2 will support OCT leads facilitating the OCT. We will identify barriers and successes as we dig deeper into the craft of evaluation and the improvement of inter-rater agreement.


Registration Link:

https://attendee.gotowebinar.com/register/4263554616508740097


Contact:

Kimberly Simmons - kimberly.simmons@dpi.nc.gov

OCT: Check-in Webinar

Tuesday, March 1st, 4-5pm

This is an online event.

Intended Audience: OCT Participants



Please register for OCT Check-in Webinar on Mar 01, 2016 4:00 PM EST at:

https://attendee.gotowebinar.com/register/3607820177828870402

Information will include updates, support information and resources based on participation and performance.

Contact:

Kimberly Simmons - kimberly.simmons@dpi.nc.gov

OCT Wrap Up Webinar

Thursday, May 19th, 4pm

This is an online event.

Target Audience:


OCT Participants


Description:


Please register for OCT Wrap up Webinar on May 19, 2016 4:00 PM EDT at:


https://attendee.gotowebinar.com/register/7333300923780603650


The final webinar will examine areas of improvement in calibration across the entire group of evaluators who participated in the two pilot scoring studies. It will also explore how reports provided by Observation Engine can be used to customize professional development and support collaborative learning opportunities in your LEA.



After registering, you will receive a confirmation email containing information about joining the webinar.


Contact:


Kimberly Simmons

kimberly.simmons@dpi.nc.gov

Have questions about Observation Calibration Training?

If you have questions regarding the OCT, please contact Kimberly Simmons, North Carolina Educator Evaluation System Consultant.