SUBJECT CHARACTERlSTlCS
The selection of people for a study may resu1t in the individuals (or groups) differing from one another in unintended ways that are related to the variables to be studied. This is sometimes referred to as ”selection bias,” or a subject characteristics threat. In our example of teacher expectation and class disruptive behavior, the ability level of the class fits this category. In studies that compare groups, subjects in the groups to be compared may differ on such variables as age, gender, ability, socio- economic background, and the like. If not controlled, these variables may ”explain away” whatever differences between groups are found. The list of such subject characteristics is virtually unlimited. Some examples of subject characteristics that might affect the results of a study include the following:
Age, strength, maturity, gender, ethnicity, coordination, speed, intelligence, vocabulary, attitude, reading ability, fluency, manual dexterity, socioeconomic status, religious beliefs, political beliefs....
LOSS OF SUBJECTS (MORTALITY)
No matter how carefully the subjects of a study are selected, it is common to ”lose” some as the study progresses. This is known as a mortality threat. For one reason or another (for example, illness, family relocation, or the requirements of other activities), some individuals may drop out of the study. This is especially true in most intervention studies, since they take place over time. Subjects may be absent during the collection of data or fail to complete tests, questionnaires, or other instruments. Failure to complete instruments is especially a problem in questionnaire studies. In such studies, it is not uncommon to find that 20 percent or more of the subjects involved do not return.
HISTORY
On occasion unplanned-for events could occur during the course of a longer term study which can affect the responses of the subjects. Such events are referred to as the history effect. For example, a visitor who dropped by or a sensational media event which may affect the participants that day or over a period of time. For example, stress levels may alter during a period of economic or personal difficulty affecting other responses.
LOCATION
The particular location in which data are collected or in which an intervention is carried out, may create alternative explanations for results. This is called a location threat. For example, classrooms in which students are taught may have more or less resources, workstations, lighting, or teachers who may skew the results inadvertently. The location in which tests are adminstered may affect responses. Parent assessments of their children may be different when done at home than at school or if done individually or in groups. Student performance on tests may be lower if tests are given in noisy or poorly lighted rooms. Observations of student interaction may be affected by the physical arrangement in certain classrooms. The best method of control for a location threat is to hold location constant that is, keep it the same for all participants.
TESTING
In intervention studies, where data are collected over a period of time, it is common to test subjects at thc beginning of the study. By testing before and then testing after the experiment is done, one can measure changes in results due to the intervention. However, there is also a tesing threat in that the participants may be alerted during the pre-test as to the areas in which they should pay more attention during the experiment.
INSTRUMENTATION
Instruments are devices used by researchers to collect information. Examples are: questionnaires, surveys, tests, observation, participation, studies….
Instrument Decay can be a problem if the nature of the instrument is changed over time. This may be due to fatigue or repetition on the part of the person adminstering the test, taking the test, or correcting the test.
Data Collector Characteristics is an inevitable part of most instruments and can affect results. The individual who collects the data may affect the results inadvertently. For example, people may be more willing to be interviewed by females rather than males. Other characteristics could be language patterns, ethnicity, age, size….
Also, individuals may present information, researchers may collect data differently, or counsellors may use different tactics when presenting orally. These threats, know as the implementer effect neede to be controlled for as much as possible.
MATURATION
Changes may occur during a study which are due to factors associated with the passing of time rather than the intervention itself. This is known as the maturation threat. Over the course of a semester, for example, young or teenage students in particular will change in many ways due to aging and experience.
ATTITUDE OF SUBJECTS
The way in which subject view a study and their participation in it can create a threat to internal validity. One example is the well known ”Hawthorne effect” first observed in the Hawthorne plant of the Western Electric Company some years ago.’ It was accidentally discovered that productivity improved not only when improvements were made in physical working conditions (such as an increase in the number of coffee breaks and better lighting), but also when such conditions were unintentionally made worse (for instance, the number of coffee breaks were reduced and the lighting was dimmed). The usual explanation for this is that the special attention and recognition received by the workers were responsible; they felt someone cared about them and was trying to help them. This increased attention and recognition of subjects has subsequently been referred to as the Hawthorne effect. An opposite effect can occur whenever, in intervention studies, the members of the control group receive no treatment at all. As a result, they may become demoralized or resentful and hence perform more poorly than the treatment group. It may thus appear that the experimental group is performing better as a result of the treatment when this is not the case. It has also been suggested that recipients of an experimental treatment may perform better due to the novelty of the treatment rather than because of the specific nature of’ the treatment, It might be expected, then, that subjects who know they are part of a study may show improvement as a result of a feeling that they are receiving some sort of special treatment – no matter what this treatment may be.
REGRESSION
This threat may be present whenever change is studied in a group that has been selected to be part of a group due to due to their extreme scores on the dependent variables. After being exposed to some sort of instrument, the extreme individuals will be more likely to show more improvement (or less improvement) than an average subject and therefore the results will seem to be more extreme.
From: Research Methods in Social Sciences
D. Nachmias, 1981
[Index]