Literature Reviews

Lab 6

Instructor: Emma Marshall, JD, MA

Today’s Tasks

• Today’s lab:
  • How to read/interpret Research
  • Lit Review Proposal (some things to think about as you read)

Review: Research Critiques

1 Hypotheses.

• Look for all statements about the research hypothesis.
  • Remember can be explicit or implied

2 External validity components.

• What looks good? What raises red flags? What is unclear?
  • Relate generalizability back to stated goals of study & pop of interest

3 Internal validity components.

• What looks good? What raises red flags? What is unclear?
  • Relate confounds or causal interpretation back to stated goals of study & design/hypotheses

Review: Research Critiques

4 Anything else of interest you find!

• Measurement/Construct validity
  • How did they operationalize and measure key concepts?
• Statistical Conclusion validity
  • Can findings replicate?
  • What about effect sizes?

5 Review your annotations and brainstorm alternatives.

• Make suggestions or identify gaps in knowledge, do not always want to change the purpose, hypothesis, or overall design.

Review: Validity

Review: Variables

Practice: Research Designs

Link to watch

Pobodies Nerfect!

This is copied over from the lecture section, week 3. Go through this as a refresher of what kinds of validity there are, and what these mean for interpretation of a study.

Practice: Research Designs

Have you ever watched dance challenge videos on TikTok? If so, you probably noticed the skill and appearance of the dance model. Some psychologists have started to study how TikTok dance models affect viewers. A set of three studies asked, can the size of the dancer on TikTok influence the body satisfaction of a viewer? Journalists at PsyPost wrote up a summary of this empirical work. The research was originally published in the journal, Computers in Human Behavior; may be paywalled.

Find the article

What is the theory

The theory behind the study is related to the potential of social media to impact body esteem:

• “Media, especially its portrayal of the”thin ideal” where extremely slim bodies are considered attractive, has a significant impact on shaping these negative body perceptions…”
• “With the rise of social media platforms like TikTok, researchers are curious about how this new form of media might influence people’s body image…”

Study 1: Overview

• Researchers recruited 262 women aged between 18 and 25 years for an online experiment.
• Ps randomly assigned to watch TikTok videos featuring:
  • thin dancers
  • large dancers
  • videos of animals (control)
• Both before and after watching the videos, participants rated:
  • weight satisfaction
  • overall appearance satisfaction
  • body shape satisfaction

Study 1: What type of Design?

Study 1: What were the Variables?

• Variable Name
• Levels or scale?
• Manipulated or measured?
  • IV or DV or ?? If IV, WG or BG?

Study 1: What were the Variables?

Study 1: What are the Research Hypotheses?

Study 1: Results

• Body satisfaction increased after large dancers and control (animal) videos
  • But decreased after thin dancer videos

Study 1: Are these results generalizable? Are they interpretable?

Purpose of Study 2

• Replicate study 1 results
• Identify why IV ➭ DV relationship
• Task/stimulus

Study 2: Task/stimulus changes?

Internal or External validity?

Study 2: Variables changes?

Internalization…?

Study 2: Variables

Study 2: Results?

A majority of participants in the thin dancer condition engaged in upward comparison (comparing themselves negatively to the dancers), while a majority of participants in the large dancer condition engaged in downward comparison (comparing themselves positively to the dancers).

Study 3: Purpose?

Warning

• Study 1 & 2:
  • Ps aware study about body satisfaction…
  • Used pretest-postest design…
• Demand characteristics!!
  • Internal or External validity issue?

One potential internal validity threat to both Study 1 and Study 2 is that participants were told the study was about body satisfaction. In addition, the pretest-postest nature of the design may have nudged participants to report changes in line with the study’s goals.

Study 3: Used post-test-only design

Warning

Demand characteristics can be a threat to either INTERNAL or EXTERNAL VALIDITY!

• Ps aware study about body satisfaction…may change behavior/answers!
• Used pretest-postest design…testing effects!

• Why use a posttest-only design to control for these validity threats?

Study 3: Design

• Recruited 375 women aged between 18 and 25 years for an online experiment
• Ps randomly assigned to watch either TikTok videos featuring:
  • thin dancers
  • large dancers
• Posttest-only design
• Told ps study about how engaging different TikTok videos are

Study 3: Task/Stimulus and Procedures

Study 3: Results

After watching the videos, the participants completed the same measures of body satisfaction used in the previous studies.

Even after concealing the true purpose of the study, the results consistently indicated that watching videos of thin dancers led to lower body satisfaction compared to videos of large dancers.

What do you think of this set of studies?

• How does it do with four types of Validity?
  • Internal Validity - how does it rule out threats?
• External Validity
  • samples/task/generalizability?
• Measurement/Construct
  • how does it measure the key concepts?
• Statistical Conclusion
  • replicate? effect sizes?

• Specifically, when you address statistical validity, use terms like replication and effect size.

• When you address internal validity, think about how the study rules out most internal validity threats.

• When you address construct validity, ask a question about the measurement of the dependent variables.

• When you address external validity, think about the samples used in these studies.

Generalizing results?

Here’s how the researchers concluded an interview about their studies:

“The findings also raise the possibility that TikTok could change its algorithm and show a wider variety of body types, and this could have a major public health benefits. Body dissatisfaction is major risk factor for eating disorders and eating disorders have the highest mortality rate of all psychiatric diseases due to medical complications or suicide…”

Can we generalize to the algorithm on a users’ feed?

Limits of generalization

Sample ➡️ Population?

Setting/Task/Stimulus