Research Methods study guide

Applied research
Conducted to address issues in which there are practical problems and potential solutions.
1. Reliable or trustworthy sources
2. News media
3. Books
4. Government officials
5. Religious figures
6. Political pundits
Basic Research
Attempts to answer fundamental questions about the nature of behavior
Knowledge is based on observations. Data (observations) are collected that form the basis of conclusions about the nature of the world
Good scientific ideas are testable. Research can either support or falsify them
Relies on personal judgement. Example: this ad should persuade them to quit smoking.
We will write a custom essay sample on
Any topic specifically for you
For only $13.90/page
Order Now
Definitely not a science. Hypotheses generated are not typically testable. Example: a pill that claims to improve memory…or increase weight loss. claims tend to be vague and ignore conflicting evidence
Discussion section
Reviews the research from various perspectives. Compares with past results. Includes suggestions for practical applications and for future research on the topic. (strengths and weaknesses)
Introduction Section
1. Outlines problems
2. Past research and theories
3. Formal hypotheses or specific expectations
It is a tentative statement about the relationship between two or more variables. It is a testable prediction about what you expect to happen in your study.
APA Ethics Code
5 General Principles
1. Beneficence
2. Responsibility
3. Integrity
4. Justice
5. Respect for the rights and dignity of others
The Belmont Report
Ethical Principles and Guidelines for the Protection of Human Subjects of Research
Justice (Belmont Report)
Achieved through selection of persons
Beneficence (Belmont Report)
Achieved through assessment of risks and benefits
Autonomy (Belmont Report)
(Respect for persons) – achieved through informed consent
Happens after the study. Tells them the purpose of study, anticipated results, practical implications.
Exempt Research
Observations. No manipulation of the subject. Example: anonymous surveys, questionnaires, educational tests
Honest Experiments
Participants are made aware of the purpose of the research and that they will be studied (e.g., speed dating studies; study skills improvement program)
Informed Consent
The purpose is to provide potential participants with all information that might influence their decision about whether to participate
Minimal Risk Research
When the risk of harm is no greater than risk encountered in daily life or routine physical or psychological tests. Example: physiological data, moderate exercise.
Routine review conducted by the IRB
Confounding variable
When an uncontrolled third variable is operating and is not wanted. Changes results of DV. Example: Activity level (IV) weight gain(DV) age (CV)
Construct validity
Concerns whether our methods of studying variables are accurate.
Example: Think about the bathroom scale—is this an accurate measure of weight?
Internal Validity
is the extent to which you are able to say that no other variables except the one you’re studying caused the result. For example, if we are studying the variable of pay and the result of hard work, we want to be able to say that no other reason (not personality, not motivation, not competition) causes the hard work. We want to say that pay and pay alone makes people like Sean work harder.
External Validity
Concerns whether we can generalize findings of a study to other settings/places/populations.
Correlation coefficient
The way of measuring the strength of the linear association between two variables.
Ranges from r = -1.0 to +1.0
Closer to 1 (+ or -) = stronger relationship between variable
Curvilinear relationship
As scores on one variable increase, scores on the second variable tend to increase, then decrease, (then increase)
Experimental Control
All extraneous variables in an experiment are kept constant; so that it cannot be responsible for the results of a study—in other words, it cannot be the confounding variable.
Experimental control is accomplished by treating participants in all groups in the experiment identically; the only difference between groups is the manipulated variable.
Experiment Method
One variable is manipulated or controlled and the other is measured (REMEMBER: in the NONexperimental method, both variables are measured). Attempts to eliminate the influence of all potential third variables on the dependent variable
Field Experiment
Researchers manipulate an IV in a natural setting, such as a parking lot or a mall. Example: researchers have studied people’s parking behavior by driving a car in a variety of ways
Negative Linear Relationship
Higher scores on one variable tend to predict lower scores on the other variable
Positive Linear Relationship
Higher scores on one variable tend to predict higher scores on the other variable OR Lower scores on one variable tend to predict lower scores on the other variable
Non-experimental method (correlational method)
Relies on observation or interactions to come to a conclusion. The non-experimental researcher must rely on correlations, surveys or case studies, and cannot demonstrate a true cause-and-effect relationship. Cannot manipulate variables. Both variables are measured.
Operational Variable
The specific way in which a variable is measured in a particular study
Participant Variable
The differing individual characteristics of participants in an experiment. Example: age, background, socioeconomic status, current mood.
The researcher can be confident that the characteristics of the participants in the two groups will be virtually identical.
Third Variable Problem
There is danger that no direct causal relationship exists between the measured variables. X and Y appear to be related. But they do not influence each other. Instead, a third variable (T) causes both of them to change.
Concurrent Validity
Examining the relationship between a measure and a behavior at the same time

Example: studying a measure of shyness in a group of salespeople (probably not shy) and IT experts (probably shy) in the same company

Content Validity
Compare measure with other content that defines the variable.
Example: A measure of depression should have content that links to each of the symptoms that define depression
Convergent validity
Extent to which scores on a measure are related to scores on other measures. Measures of similar constructs should converge. Example: One measure of shyness should correlate highly with another shyness measure or a measure of social anxiety
Cronbach’s Alpha
Correlation of each item on the measure all other items on the measure. Ranges from 0.00 (no reliability) to 1.00 (perfectly reliable). α = .80 or greater indicates high reliability
Discriminant Validity
Scores on a measure are not related to variables with which they should not be related. Measure should discriminate between the variable of interest and other unrelated constructs Example: Shyness and value of forcefulness with others should not be related
Face Validity
whether or not your study measures what it is supposed to measure. You can think of it as where you just skim the surface in order to form an opinion.
Internal Consistency Reliability
Extent to which raters agree in their observations. Single observation of one rater may be unreliable. Solution is to use multiple raters to observe the same behavior. Cohen’s Kappa: Correlation between the observations of raters
Interval Scale
Difference between numbers on the scale are meaningful. no absolute zero
Example: thermometer
Item-Total Correlation
The item total correlation is a correlation between the question score (e.g., 0 or 1 for multiple choice) and the overall assessment score (e.g., 67%)
Measurement Error
Nominal Scale
No quantitative information. categories.
Ordinal Scale
Allow us to rank order the levels of the variable being studied. No particular value is attached to the l ls between numbers
Example: Star ratings for movies
Pearson Product-Moment Correlation Coefficient
Is a measure of the linear correlation between two variables X and Y, giving a value between +1 and −1.
Predictive validity
Using scores on one measure to predict future behaviors. Example: Validity of SAT test is demonstrated by its ability to predict performance in college
Ratio Scale
Just like interval, an absolute zero indicates the absence of whats being measured.
Example: weight
Measure is reactive if awareness of being measured changes peoples’ behavior. A reactive measure tells us how people behave when they know they are being measured, but not how they would behave under natural circumstances. is a phenomenon that occurs when individuals alter their performance or behavior due to the awareness that they are being observed.
Split-half reliability
A measure of reliability in which a test is split into two parts and an individual’s scores on both halves are compared.
Test-retest reliability
Obtained by administering the same test twice over a period of time to a group of individuals. Example: if a group of students take a geography test just before the end of semester and one when they return to school at the beginning of the next, the tests should produce broadly the same results.
Closed-ended questions
Limited number of response alternatives given. More structured. Response alternatives the same for everyone. Example: how much do you dislike or like this class?
Graphic Rating Scale
A graphic rating scale is a commonly used scale system for performance appraisals. The scale typically features a Likert scale from 1-3, 1-5, and so on. An example of a 1-3 rating could include responses such as: 1: Poor, 2: Average, and 3: Excellent.
Open Ended Questions
Respondents are free to answer any way they would like. Require time to categorize and code
Rating Scale
Rating Scale
Ask people to provide “how much” judgments on any number of dimensions
Example: Amount of agreement, liking, or confidence
Semantic Differential Scale
Semantic Differential Scale
Respondents rate any concept (people, objects, behaviors, ideas) on a series of bipolar adjectives
Dillman’s (2008) Principles
1. Simplicity
2. Negative wording
3. No Loaded questions
4. No Double-barreled questions
5. Nay Yae questions
(dropout factor) Even if groups started the same, dropout may cause them to become different. Pretest allows us to assess whether dropout made groups different. Most likely in longitudinal studies
Between-subjects design (also independent groups design)
Subjects participate in only one group – either experimental or control. Comparisons made between different groups of subjects. Random assignment.
Carryover Effect
Possible that the effect of the first condition carries over and influences the response to the second condition
Confounding variable
Confounding variable
Another variable that occurs along with the independent variable . Is an uncontrolled variable. Cannot determine which variable is responsible for the effect. Good experimental design requires eliminating confounding variables
Fatigue effect
Change of performance on second task due to tiredness, boredom, or distraction
Matched Pairs Design
People are matched on a participant characteristic. Matched to either the dependent measure or a variable that is strongly related to the dependent variable
Example: GPA, weight, diet.
Posttest-only design
Obtain two equivalent groups of participants. Introduce the independent variable. Measure the effect of the independent variable on the dependent variable
Practice effect (also learning effect)
Practice effects can be defined as influences on performance that arises from a practicing a task. Even after, participants have a tendency to perform initial trials poorly because they are still not warmed up to it
Pretest-posttest design
Only difference: A pretest is given to each group prior to introduction of the experimental manipulation. Assures that groups are equivalent at the beginning of the experiment. Not usually necessary with random assignment to conditions
Repeated measures design (also within-subjects design)
The same people participate in both conditions. Example: If you wanted 10 participants in each condition, you would need 10 participants total
Selection Differences
The difference between the average value of a quantitative character in the whole population and the average value of those selected to reproduce the next generation
Example: The selection differential is the difference of the base population mean and the mean of the selected parents. The selection response is how much gain you make when mating the selected parents.
Factorial Design
Experiments with more than one IV (or factor). Simplest factorial design: 2 X 2 factorial design. Has two IVs, each IV has two levels. Example: “Gender” might be a factor with two levels “male” and “female” and “Diet” might be a factor with three levels “low”, “medium” and “high” protein.
Exists when the effect of one IV on the DV depends on level of the other IV. Can’t be obtained in simple experimental designs that only have two levels
Mixed Factorial Design
Mixed Factorial Design
involves two or more independent variables, of which at least one is a within-subjects (repeated measures) factor and at least one is a between-groups factor. In the simplest case, there will be one between-groups factor and one within-subjects factor.
Main Effect
The effect of each IV taken by itself. In a design with 2 IVs, there are two main effects – one for each IV. Tells you about the overall relationship between an IV and the DV