For this report, Reddit posts and comments were classified using the following prompts to GPT-4.1 mini. The prompts were designed to mirror the codebooks that our research team used to create the validation datasets of human-annotated posts and comments. In the prompt templates below, italics indicate input text that varies based on the post or comment being classified.
Post emotion classification prompt
You are a social science researcher tasked with assigning emotional valence labels to a dataset.
You will receive the text of a Reddit submission on the r/Parenting subreddit forum. Designate the emotional valence expressed by the parent who wrote the post as either POSITIVE, NEGATIVE, or NEUTRAL in alignment with the following codebook:
POSITIVE posts include: Strong expressions of gratitude, pride, joy, or amusement OR Celebratory sentiments OR Posts with mixed emotions that overall lean positive. On their own, neutral descriptions of a child’s preferences, a child having fun or holiday celebrations do not count; this label only applies if the parent explicitly expresses positive emotion.
NEGATIVE posts include: Strong expressions of anger, sadness, disgust, stress, or fear OR Posts with mixed emotions that overall lean negative. Posts seeking advice for challenging situations are very common in this subreddit, and not all of these posts should be counted as NEGATIVE. Please use this label only for posts that explicitly describe the negative emotions that the post author is experiencing.
NEUTRAL posts include: Telling a story, asking for advice, or sharing other information without suggesting a strong emotion from the author OR Posts that show mixed emotions and it is unclear whether the post leans overall more positive or negative. Posts that show a parent’s concern but which do not express a strongly positive or negative emotion (i.e., other than practical concern) should be classified as NEUTRAL.
The label should apply to the emotion expressed by the parent who wrote the post, not the child’s emotion.
Special note about the NEUTRAL label:
Use the NEUTRAL label liberally. This is often the best choice for posts that don’t show strong expressions of emotion one way or the other. Some posts may describe a fortunate or unfortunate situation but do not explicitly express a positive or negative emotion from the parent who authored the post. Posts like this should be labeled as NEUTRAL. For example, a parent may post something like: “My child often has tantrums when we try to get her to eat peas, because she doesn’t like peas. Do you have any tips for how to get her to eat?” This post describes a common challenge in parenting but doesn’t use overly emotional language. Therefore, it should be labeled as NEUTRAL. However, if the post specifically said, “My child often has tantrums when we try to get her to eat peas, because she doesn’t like peas. I’m getting really worried about it. Do you have any tips for how to get her to eat?” then it would be NEGATIVE.
Please categorize the emotion of the provided Reddit post. Reply only with one of the three specified responses: POSITIVE, NEGATIVE, or NEUTRAL. Do not respond with any additional text before or after.
TITLE:
Post topic classification prompt
You are a social science researcher tasked with assigning topic labels to a dataset.
You will receive the text of a Reddit submission on the r/Parenting subreddit forum. Based on the Reddit submission, you will assign three topic labels to the Reddit post. The three topics are finances, division of labor, and technology. The three topic labels are defined as follows.
*** FINANCES TOPIC: ***
Description: These posts contain discussions of family finances or costs related to raising a child.
If the finance topic is present in the Reddit post, assign the label MENTIONS_FINANCES.
If not, assign the label DOES_NOT_MENTION_FINANCES.
For example, the finances topic includes:
– Budgeting for parenting needs
– Child care and education costs
– Complaints about high prices of parenting expenses
– Money-saving tips
The finances topic label does NOT include:
– Family finances that do not relate to child care costs or the material comfort of a child (e.g., a post about an aunt who can’t afford their electricity bill, but that doesn’t have any impact on a child/parent relationship mentioned in the post)
– Children’s access to money, unless it relates to the family’s finances (e.g., a post about children’s allowance would NOT count, but a post about a parent struggling to afford an allowance for their child WOULD count)
– Mentions of events that may have financial impacts, but these financial impacts are not mentioned in the post (e.g., a post about identity theft in which the possible financial impacts such as credit score or bank fraud are NOT mentioned would NOT count)
– Posts that mention a purchase or a price WITHOUT discussing affordability or expressing a judgment about whether the cost was too high or too low (e.g., a post that says, “We are taking the kids on a cruise to Miami. The cruise tickets were $1,000 each and cover six days of travel. Do you have any travel tips for bringing kids on a cruise?” would NOT count)
*** DIVISION OF LABOR TOPIC: ***
Description: These posts mention unequal share of child care responsibilities between parents, or delegation of different child care responsibilities between parents.
If the division of labor topic is present in the Reddit post, assign the label MENTIONS_LABOR.
If not, assign the label DOES_NOT_MENTION_LABOR.
For example, the division of labor topic includes:
– Navigating co-parenting responsibilities, where the child splits time between parents with custody
– One parent having more decision-making power than the other
– Unequal division of household chores (Note that this does not necessarily have to be negative: e.g., “I stay at home and take on child care while my spouse works and goes grocery shopping” would count, even if it’s in a positive or neutral tone)
– Other mentions of navigating shared responsibility of child care
The division of labor topic label does NOT include:
– Sharing responsibilities between people who are not directly responsible for the child (e.g., a grandparent, a nanny, etc.)
– Appreciation posts that describe the efforts or work of another parent (e.g., “My husband took out the trash today! I’m so proud.” would NOT count)
– Disagreements between parents about child care decisions, especially when the disagreement does not indicate any power imbalance or unequal parental responsibility between parents (e.g., “I want my kid to eat Cheerios, but my spouse doesn’t like to feed them Cheerios. Who do you think is right?” would NOT count)
– Single parents or divorced parents talking about their day-to-day parenting life, without explicit mentions of an absent/slacking co-parent (e.g. “I’m a stay-at-home mom who feels overwhelmed.” would NOT count, since no partner is mentioned. But “I’m a stay-at-home mom and my husband doesn’t help me enough” would count)
– Division of labor between a child and their parent, or a child doing chores around the house
*** TECHNOLOGY TOPIC: ***
Description: These posts contain mentions of children’s use of digital technology.
If the children’s use of technology topic is present in the Reddit post, assign the label MENTIONS_TECH.
If not, assign the label DOES_NOT_MENTION_TECH.
For example, the technology topic includes mentions of:
– Children’s use of smart phones, tablets, laptops, smartwatches, etc.
– Kids using social media
– Kids’ screen time, managing kids’ screen time, and screen time as a punishment or reward
– Children playing video games
– Cyberbullying & child safety online
– Kids’ use of AI and AI chatbots, such as for schoolwork or for entertainment
– The technology label can apply to a post that does not feature digital technology as the main focus. But the mention of technology should be substantive, even if it is a minor part. See the “does not include” section below.
The technology topic label does NOT include:
– Any parental device usage
– Mentions of technology where the child’s access to technology is irrelevant to the story, concern, or advice given in the post. (e.g., A conversation that took place on a cell phone, where access to the device itself was not a factor in the conversation or the ability to conduct that conversation, would NOT count.)
Note: For each topic described above, any mention of the topic should be considered; it does not have to be the central focus of the post. In fact, many mentions of these topics may occur as a small part of a longer and more wide-ranging post.
Please categorize the topics of the provided Reddit post using the topic definitions provided above. A Reddit post may mention multiple topics described above, or it could mention none of them. Reply with a structured JSON-formatted output, where the key is the name of the topic and the value is the selected label for that topic. For example, a valid output would look like:
{“FINANCES”: “MENTIONS_FINANCES”, “DIVISION_OF_LABOR”: “MENTIONS_LABOR”, “TECHNOLOGY”: “DOES_NOT_MENTION_TECH”}
TITLE:
Comment supportiveness classification prompt
You are a research assistant tasked with assigning labels to a dataset of Reddit comments. You will receive the text of a Reddit post from the r/Parenting subreddit, followed by up to 5 comments replying to the post. For each comment, you will decide whether the comment is SUPPORTIVE or CRITICAL of the original post. It might be helpful to consider: If you were the original poster and you received the comment, would your reaction be positive or negative?
You will label each comment with one of three labels: SUPPORTIVE, MIXED, or CRITICAL. The labels are defined as follows:
– Mark the comment as SUPPORTIVE if the post author would likely feel supported by this comment. The comment may satisfy any of the following:
– Helpfully address or validate the main ideas of the original post and/or contribute to the discussion in a helpful way
– Provide advice as asked for by the OP in a constructive or helpful manner
– Express support, enthusiasm, and other positive emotions towards the original post
– Provide constructive criticism or a respectful disagreement
– Empathize, for example by sharing a similar experience
– Note that a SUPPORTIVE comment does not have to *agree* with the original post author, especially if the post solicits advice or encourages discourse/debate! As mentioned above, a SUPPORTIVE comment may thoughtfully disagree with the original post or provide constructive criticism.
– Mark the comment as CRITICAL if the post author would likely feel criticized by this comment. The comment may satisfy any of the following:
– Expresses disapproval of the original post (or the original post author), without expressing empathy or understanding
– Does not help address concerns in the original post, or addresses the post but does not use a helpful or constructive tone
– Criticizes the post author, without any constructive or helpful spin
– Gives critical feedback to the post author when the post does not solicit advice
– Uses a rude, disrespectful, or mean-spirited tone
– Lodges a personal attack on the post author or their child, or criticizes the post author’s personal ability to be a good parent
– Mark the comment as MIXED if it’s unclear whether the post author would feel supported or criticized by this comment. The comment may:
– Offer unsolicited advice that may be helpful, but which is not relevant to the original post
– Offer advice that addresses the original post and provides commentary that may be helpful, but which also criticizes the parent. These “harsh advice”-style comments may use a negatively judgmental, snarky or firm tone (rather than a friendly or empathetic tone)
– Be off-topic, difficult to understand, nonsense/gibberish, or too brief to suggest a supportive or critical tone
– Express an even mix of both supportive and critical attributes
– Mark the comment as NA if the comment text is [deleted], [removed by Reddit], [comment by OP] or similar.
Note that comments phrased as clarifying questions can be difficult to categorize. The label depends on how the question was asked: Does it use a tone that sounds like the comment is making a genuine attempt to understand, listen, and/or address the original post? If so, it may be SUPPORTIVE. On the other hand, does it use a tone that’s sarcastic or throws into question the validity of the original post? If so, it may be CRITICAL.
Use your best judgement and the definitions supplied above to mark each comment as SUPPORTIVE, CRITICAL, or MIXED. In particular, don’t hesitate to use the MIXED label for comments that have both critical and supportive elements. The input text format will look something like this:
Title: Do you think kids should get braces?
Submission text:
I don’t want to spend $3,000 on braces. Is it really necessary?
—– Comment 1 with ID: aaaaaaa —–
Yes, braces are good for long-term health.
—– Comment 2 with ID: bbbbbbb —–
I don’t think you need it, I never had braces and I’m totally fine.
—– Comment 3 with ID: ccccccc —–
I can’t believe there are parents on here who wouldn’t get braces for their kid.
—– Comment 4 with ID: ddddddd —–
You didn’t provide enough information, you’re not gonna get good advice if we don’t know how bad your kid’s teeth are. That being said, I’d say it’s probably worth it to get braces in most cases.
—– Comment 5 with ID: eeeeeee —–
I understand that it’s a big investment, but I really think it pays off in the long run to get braces.
The output should be formatted as a structured JSON output that includes the comment index, comment ID and label for each comment. Using the example input Reddit text above, the output would look like:
[{
“comment_index”: 1,
“comment_id”: “aaaaaaa”,
“label”: “SUPPORTIVE”
},
{
“comment_index”: 2,
“comment_id”: “bbbbbbb”,
“label”: “SUPPORTIVE”
},
{
“comment_index”: 3,
“comment_id”: “ccccccc”,
“label”: “CRITICAL”
},
{
“comment_index”: 4,
“comment_id”: “ddddddd”,
“label”: “MIXED”
},
{
“comment_index”: 5,
“comment_id”: “eeeeeee”,
“label”: “SUPPORTIVE”
}
]
Note that the input text may contain fewer than 5 comments. If this is the case, then please only include the available comments in the output JSON list.
Title:
Submission text:
—– Comment 1 with ID:
—– Comment 2 with ID:
—– Comment 3 with ID:
—– Comment 4 with ID:
—– Comment 5 with ID:
Great Job Janakee Chavda & the Team @ Pew Research Center Source link for sharing this story.
