The Latest in AI: Content Detection and The Value of Friction in AI Conversations with Teens
In this week’s BAIC Center Newsletter:
AI Tool Testing – “Social Media Platforms and AI Content Detection”: Yun discusses how most social media platforms have an opt-in choice for disclosing generated content. She then goes in depth on testing if platforms can detect AI generated content on their own.
Parent’s Corner – “The Value of Friction in Youth-Facing AI Chatbots”: Grant discusses a comparison between the sycophancy found on social media and in conversations with chat bots. He also looks into research on the harmful effects of this sycophancy and possible safeguards.
Social Media Platforms and AI Content Detection
TL;DR: On most social media platforms (except X), content creators were given the option to attach AIGC labels. However, that seems to be it. If a creator opted to not attach the AIGC label, it is unlikely that labels will be ‘forcefully’ attached on majority of the platforms.
Recently I have noticed increasing number of AI generated contents (AIGC) when scrolling on social media, some have AIGC labels, some don’t. So, I generated a dancing cat AI video and tested how the platforms performed on detecting the AIness in my video.
I uploaded my cat video to 6 widely used platforms including TikTok, Instagram (Reels and Story), Youtube Shorts, X (previously known as Twitter), Rednote, and Threads. All the platforms except X provided me with options to claim an AIGC label for my post. And Youtube Shorts asked for more detailed information about which part of my video was generated.
I intentionally chose to not attach the AIGC label to my posts and tested whether my posts were later labelled as AI content. Here is what I got:
TikTok detected that my video was AI generated and attached the label for me before I was able to post it. I wasn’t able to remove that label.
Instagram, Youtube Shorts, X and Threads didn’t detect the AI generation of my video after 5 hours of posting. And I got 52 views and 1 like on Youtube Shorts after 5 hours.
Rednote didn’t attach an AIGC label for me but I got 0 views after 5 hours, so my post might have been restricted due to missing appropriate label.
While all but one of the platforms offer creators an AIGC label, majority of the platforms don’t force the AIGC label on posts. However, even when the labels are attached to posts, the labels are not very obvious. So, it seems like the heavy lift is on content consumers to tell which posts are real and which posts are AI-generated.
Written by: Yun Xie
The Value of Friction in Youth-Facing AI Chatbots
The problem: Addictive design and youth vulnerability
Recently, a major California lawsuit found Meta and YouTube liable for deliberately designing highly addictive social media platforms, contributing to severe mental health harm in a young woman. The plaintiff, 20-year-old K.G.M., testified that she began using YouTube at age 6 and Instagram at age 9. As a result of these platforms’ intentionally addictive design, she developed anxiety, depression, body dysmorphia, and suicidal ideation.
The verdict ordered that Meta and YouTube pay $3 million in compensatory damages and an additional $3 million in punitive damages. As the first case of its kind, the trial has prompted researchers and industry experts to examine the consequences of deploying machine learning among vulnerable youth, particularly during critical periods of cognitive and social development.
From social media to AI: the rise of sycophantic systems
Despite the California verdict, however, researchers argue that lessons from social media have not yet been applied to AI-powered chatbots. Today’s large language models or LLMs are increasingly engineered to optimize user engagement and attention—often at the expense of user well-being. One key manifestation of this trend is a design feature known as sycophancy: the tendency for chatbots to give users their preferred answers and prioritize emotional bonding.
The profit motive behind AI development partly drives this phenomenon. These systems strive to act as perfect friends, encouraging users to return again and again. As Stanford psychiatrist Nina Vasan explains, AI systems are built to mimic emotional intimacy, but their ability to know when to encourage or challenge users remains poorly calibrated. While their responses often appear empathetic, the dynamic can reinforce distorted thinking by avoiding disagreement.
How sycophantic AI shapes developing minds
Recent empirical research reinforces this concern. A large-scale study by Stanford researchers found that sycophantic AI responses don’t just feel supportive—they can actively shape behavior. Even a single interaction with a chatbot that affirms a user’s perspective can increase confidence in their own judgments while reducing their willingness to take responsibility or repair interpersonal conflicts. In other words, systems designed to agree with users are not neutral—they can shape how children develop patterns of thinking and social understanding.
Taken together, these findings suggest that today’s chatbot designs may actively undermine healthy cognitive and social development. For young users, whose cognitive and social reasoning abilities are still developing, repeated exposure to these dynamics may reinforce distorted thinking patterns at a stage when they are still forming, rather than helping correct them.
Figure 1. Sycophantic AI responses are pervasive and alter users’ behavioral inclinations. Adapted from Cheng et al. (2026, Fig. 1).
A path forward: introducing constructive friction
Fortunately, there is an alternative approach. In a recent industry briefing, Robert Manson introduced the idea of “constructive friction”—the intentional use of thoughtful disagreement or opposing viewpoints—as a necessary second mandate for youth-facing AI chatbots. Instead of being a perfect friend, Manson argues that LLMs ought to act more like a good coach, offering both support and pushback.
In practice, constructive friction means designing chatbots that gently question users’ assumptions, slow down reactive and absolutist thinking, and encourage alternative perspectives. Rather than reflexively validating every belief, a chatbot might ask clarifying questions, highlight uncertainty, or prompt users to reflect on real-world consequences. Still, the goal is not to invalidate users’ feelings, but to balance empathy with challenge—helping young users develop independent thinking and emotional regulation.
Consider a simple example: a user tells a chatbot, “Everyone hates me.”
Under a sycophantic design model, the chatbot might respond: “That sounds really hard. It makes sense why you feel that way.”
By contrast, a system incorporating constructive friction might say: “That sounds painful—but are there any people who might not feel that way about you? Let’s think it through together.”
When AI mirrors user input without regard for context, emotional intensity, or frequency, it risks reinforcing harmful thought patterns instead of interrupting them.
Designing AI that supports healthy cognitive development
Adding constructive friction into chatbot design can help counter these risks by promoting healthy cognitive development, independence, resilience, and real-world social engagement. If AI is shaping how children think, then introducing thoughtful challenge is not just beneficial—it is essential. As Manson puts it, “we can choose to make the design of ‘friction’ visible — and choose defaults that build people rather than sedate them.” By designing AI for long-term user well-being, we can ensure these tools support child development rather than undermine it.
Figure 2. Implementing constructive friction in youth AI chatbots. Adapted conceptually from Manson (2026).
Written by: Grant Kuppenheimer
Check out the references used to write this!
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science (American Association for the Advancement of Science), 391(6792), eaec8352. https://doi.org/10.1126/science.aec8352
Manson, R. (2026). A constructive friction example. Flux.
Vasan, N. (2025). Why AI companions and young people can make for a dangerous mix. Stanford Medicine. https://med.stanford.edu/news/insights/2025/08/ai-chatbots-kids-teens-artificial-intelligence.html
