Reading, Writing & Critiquing Research

Lab 5

Instructor: Emma Marshall, JD, MA

Today’s Tasks

• Today’s lab:
  • Reading & writing research reports
  • Review Lecture Material
    • Contents of a research report, Validity Net, All the words

Today is about learning to dig into a paper and read it critically. This means extracting information – especially extracting information that isn’t directly given to you in a paper and must be inferred. But, we will also learn when not to make assumptions that cover up potential flaws

Reading & Writing: Research Reports

Everything we do revolves around these six basic things!!

1. What has the author(s) read that prompted the research?
2. What is the purpose of the research being reported ?
3. What is the specific research question or research hypothesis?
4. How was the research conducted ?
5. What are the results of the statistical analysis of the data as they relate to the research question or research hypothesis?
6. How do the current findings integrate with what the author(s) read that prompted the research?

Standardizing Research with APA

• APA standardized style & formatting for journals (1928-present)
  • Currently on APA 7th Ed.
• Three main components to style-guide:
  1. Content of the report
     • Order of concepts & material
     • Rules for sections & labels
  2. Format of the report
     • margins, citations, tables, etc.
  3. Style of the report
     • language & terminology

Since there is a nearly universal set of 6 things used to describe one’s research and its outcomes, it makes sense to “standardize” those reports – making it easier to write them clearly and easier to read them.

Such “standardization” was the intent of a group of editors and business managers of anthropological and psychological journals in 1928. They wrote 7-page set of guidelines for authors. The most recent version of these guidelines, the Publication Manual of the American Psychological Association, 4th Ed., is a bit larger (368 pages), but has the same basic intent.

APA format, as we will apply it, has three components:
1) the content of the report and how it is organized and labeled 2) the format of the report (e.g., margins, citations, tables, etc.) 3) the style of the report (e.g., non-sexist language)

Elements of APA Style

Basic Rules:

• Double-space everything
• Have uniform margins of at least 1 in. at the top, bottom, right, and left of every page
• Write “formally” – without slang, contractions, etc.
• Use sentences and paragraphs (not “lists” unless hypotheses)
• Give credit to others for their ideas

Structure of an APA-style paper

APA uses six main sections that correspond with six basic components of research reports

Abstract:
50-150 word summary of the key information of study…

Introduction:
Review of the literature that prompted the research…

Method:
Participants: Who (what) and how they were selected
Materials: Equipment, measures, and materials used to collect data
Procedure: Treatment and data collection process

Results:
Description of the data analysis process and results

Discussion:
Relating the findings to the previous literature, limitations & future directions

References:
Citations of previous literature used to inform study

Introduction

General ➭ Specifics

• Background about the topic:
  • Define the topic/context
  • What does prior research say about it?
  • How has it been examined or operationalized?

• Gaps in the literature:
  • What information is missing?
  • What questions remain unanswered?

• The current study:
  • What is the gap or unanswered question you plan to study?
  • How do you plan to study it?

Method

Research recipe ➭ Ingredients + Steps

• Participants:
  • Who is your sample?
  • What is the sample size?
  • Any demographics relevant to pop of interest?

• Materials:
  • How were the variables measured?
  • How was the IV manipulated (if experiment)?
  • What are the range of values for each variable?

• Procedure:
  • What was the design?
  • Where did the experiment take place?
  • How did the experiment take place?

Results

Record of all data analysis

• Univariate statistics:
  • How is each variable distributed? Central tendency?
  • Any outliers or transformations?
  • Reliability of scales or control measures?

• Bivariate or Multivariate analyses:
  • What relationships exist between variables?
  • How did you test those relationships?

• Include all components of APA formatting for each analysis!!!!

Discussion

Summarize & contextualize findings

• Results (summarized): Brief recap of main findings without numbers

• Interpretation and Implications:
  • Were the main hypotheses supported?
  • How do your findings fit within the existing literature?

• Limitations:
  • Explain limits of interpretability/generalizability

• Future directions:
  • What are the next steps?

Where to look for information

scroll down

What has the author(s) read that prompted the research ?

Introduction, Discussion, References

What is the purpose of the research being reported ?

Abstract, Introduction, Discussion

What is the specific research question or research hypothesis ?

Abstract, Introduction, Results, Discussion

Who were the participants ?

Abstract, Method - Participants

What equipment and/or materials were used ?

Abstract, Method - Materials/Apparatus

What treatment & data collection procedures were used ?

Abstract, Method - Procedures

What are the results of the statistical analysis of the data as they relate to the research question or research hypothesis?

Abstract, Results, Discussion, Tables/Figures

How do the current findings integrate with what the author(s) read that prompted the research?

Abstract, Discussion

Beware of Abstracts

Caution

Questions?

How to critique Research Articles

• Step 1: Read (quickly-ish) the article
• Step 2: Identify basics and big take-aways (main purpose, key design, etc.)
• Step 3: Read (CAREFULLY) and take notes

Research Critiques: What to look for

1Hypotheses?

• Look for all statements about the research hypothesis
  • Remember can be explicit or implied

2External validity components

• What looks good? What raises red flags? What is unclear?
  • Relate generalizability back to stated goals of study (pop of interest)

3Internal validity components

• What looks good? What raises red flags? What is unclear?
  • Relate confounds or causal interpretation back to stated goals of study (design/hypotheses).

4Anything else of interest you find!

5Review your annotations and brainstorm alternatives

6Make suggestions or identify gaps in knowledge

• But do not change the purpose, hypothesis, or overall design! Critique what they actually did, not the ideal study in your head!