Trick or Threat

Internal and External Validity

Mirna Henriquez & Faith Oketch

Threats to Internal and External Validity

Threats of internal and external validity are factors of an unsatisfying research design.



Music Background

Dark and Stormy Night : 2 Hour Haunting Thunderstorm Sound

Threats to Internal Validity


  • History
  • Maturation
  • Testing Effects
  • Ceiling effects
  • Floor effects
  • Instrumentation effects
  • Selection bias
  • Statistical regression
  • Mortality (attrition)
  • Teacher effects
  • School effects
  • Class effects


History

The impact of events unrelated to the study that happen to take place during the study.


Maturation

Effects on study subjects due to the passage of time.


Testing effects

Effects of taking a test or questionnaire on later behavior.


Ceiling effects

A characteristic of a distribution of scores in which many scores are near the maximum possible value.


Floor effects

A characteristic of a distribution in which many scores are near the minimum possible value.


Instrumentation effects

Effects on scores due to differences in the conditions of testing.


Selection bias

Differences between groups due to different processes of selection into the groups.


Statistical regression

The tendency of extreme scores to regress toward the mean on other measures.


Mortality (attrition)

Loss of subjects over the course of a study.


Teacher effects

Effects on students of having a particularly teacher.


School effects

Effects on students of attending a particular school.


Class effects

Effects on students of being in a particular class.


Threats to External Validity


  • Lack of internal validity
  • Nonrepresentativeness
  • Artificiality
  • Reactivity
  • Hawthorne effect
  • John Henry effect

Lack of internal validity

The condition that research has not established the finding in the first place.


Nonrepresentativeness

The condition that study findings are from a setting or population unlike the one to which a researchers wishes to generalize.


Artificiality

The condition that findings of small, brief, or contrived studies may not apply to realistic settings.


Reactivity

The tendency of observation or experimentation to change the phenomenon being studied.


Hawthorne effect

A tendency of subjects in an experiment to exert outstanding efforts because they know they are in an experiment.


John Henry effect

A tendency of subjects in a control group to exert outstanding efforts because they know they are in an experiment and do not want to come out worse than the experimental group.


Activity: Name and explain the example of the scenario.

Activity: Name and explain the example of the scenario.



1. An elementary school teacher finds that her first grade students are having trouble with their basic mathematics concepts. She therefore tries a new teaching method and plans to evaluate it at the end of the year to see if it has made an improvement. She will consider her program successful if the children have mastered a large number of skills at the end of the year which they had not mastered at the beginning of the year. Which of the following is the most obvious threat to the evaluation of the program?



2. Each year an elementary school teacher provides a lesson in his physical education class on ‘The Rules of International Soccer (Football).’ In 2010, since it was a Soccer World Cup year, he decided to revise and upgrade the lesson. He initiated his lesson to coincide with the start of the televised portions of the World Cup. In the final exam, he asked his usual ten questions about International soccer. He found that the 2010 students scored substantially higher than the students from the previous three years had on the same questions. He concluded that his new program had been effective. Which of the following is the most obvious threat to the internal validity of the study?



3. Wilma's parents and teacher were concerned about her noncompliance, so they consulted a behavior therapist. They all decided together that the first order of business was to gauge the extent of the problem. For the next week, her teacher recorded all instances of the target behavior at school and her parents recorded all instances at home. Noncompliance was defined a failure to respond to an instruction within 10 seconds. The resultant data indicated that, when asked to do something, Wilma was four times more likely to obey the instruction at home than at school. They concluded that the problem was considerably worse at school, and consequently they decided to employ a different behavioral procedure in each setting.