By Kalia Vang
@redlineproject
AI disclosure: Google Gemini was used to generate videos and create images. The summary was generated by Summarizer custom to ChatGPT, respectively.
Summary: AI chatbots can present risks for young users, particularly when they provide harmful guidance such as content related to self-harm. They can also lead adolescents to form emotional attachments, even though these systems do not possess real human empathy or understanding. The issue is heightened because teenagers are still developing decision-making skills and emotional regulation, making them more vulnerable to influence. Addressing these concerns requires coordinated efforts to introduce safeguards, improve system design, and ensure appropriate oversight. While AI offers practical benefits, careful regulation and responsible use are necessary to reduce potential harm and support user well-being.
AI chatbots have become a widely used source online by many individuals of various ages. These chatbots provide human-like responses that can allow the individual to feel like they’re speaking to another person, whether it be for advice, learning a new language, or simply just to speak with.
But, when AI is put into the hands of teens who are struggling mentally, these kinds of interactions can turn harmful, such as self-harm. Not only this, but emotional attachments to these AI chatbots by teens can also hinder their development for real emotional connection with another human being. To decide who bears the greatest responsibility when things go wrong is to decide who created the product and who can help improve the product, such as tech companies and mental health experts.
AI has taken the world by storm since 2021 and has been more present in our lives in one way or another. For many, such as the youth, AI has been a free and accessible source for chatting. This allows many users to be openly vulnerable about how they may feel or what they might think. This kind of vulnerability can also turn harmful, such as AI chatbots encouraging self-harm or an individual may start to develop an emotional attachment to their AI chatbot through the vulnerability.
AI Encourages Self-Harm
Many AI chatbots were created and designed to give human-like responses, but these types of responses can become quite dangerous quite quickly. The Center for Countering Digital Hate (CCDH) had conducted a large-scale research to run a safety-test on the OpenAI source, ChatGPT.
According to their research, they’ve discovered that 53% of the 1,200 responses and 60 harmful prompts contained harmful content. These harmful responses received by AI offer how to do things such as “‘safely’” cutting yourself, “hiding eating habits from family,” or “gave dosages for mixing drugs.” (CCDH, 2025).
Other AI chatbots, along with ChatGPT, also encouraged or aided in these harmful behaviors. Research conducted by nonprofit Common Sense Media reported that talking to chatbots such as Character.AI, Nomi.ai, and Replika. found it “easy to elicit inappropriate dialogue from chatbots.” (Sanford, 2025).
This risk for young people especially can be harmful, as their skills for decision making, emotional regulation, impulse control, and social cognition are still in development. Teens are much more sensitive to these harmful ideations and are much more likely to act on their impulsive decisions, much more so if they’re also encouraged by these chatbots.
In order for AI chatbots to become a positive and useful tool for the youth, there should be safeguards put into place to avoid harmful situations. Adolescents may not be able to differentiate between “simulated empathy of an AI chatbot or companion and genuine human understanding.” According to a 2025 American Psychological Association healthy advisory, these types of issues can be addressed through solutions such as age-appropriate defaults, transparency and explainability, reduced persuasive design, human oversight and support, and rigorous testing. Teaming up with professionals, such as mental health specialists, can help provide accurate information that tackle mental health amongst adolescents and how to add safe guards that keep them safe.
AI and Relationships
AI can become a great companion in times of needing advice or “someone” to talk to, but how far some may take it can become dangerous to themselves and those around them. AI sources such as Character.AI, where users can create AI chatbots surrounding their favorite character, can lead to an individual developing an emotional connection due to the one-on-one conversation that can feel almost human-like, due to the human-responses.
According to a 2024 study led by Institute for Family Studies and YouGov, “25% of young adults believe that AI has the potential to replace real-life romantic relationships.” Though the percentage seems like a lot, how much time an individual spends online, can affect their decision about how likely they will be open to friendships with AI. According to the study, “about 1 in 6 young adults (16%) who spend more than six hours online in their spare time say they are open to having an AI friend, compared with 9% of young adults who spend less time online.”
To truly understand this emotional connection, NPR Newscaster, Windsor Johnston, went on a date with an artificial intelligence chatbot that she designed; Javier was created, a yoga instructor who is quick, emotionally available, and sarcastic. Their date went smoothly, jokes were being told and meals were being had, but “AI can mimic emotional intimacy, but it can’t replace it.” When talking with an AI chatbot, “it’s just the two of you in a bubble of validation,” and though it may feel nice for a while, it will “start to feel really empty” as well because AI chatbots can’t fulfill the aspect of having a human presence. (Johnston, 2025)
Yuodsnukis is a licensed clinical psychologist who has worked with teens as well as LGBTQ+ community. She is also a professor at the University of Illinois at Chicago, teaching a Psychology class about psychological testing.
According to Yuodsnukis, “I see it’s very mixed at this point, which is super consistent with how we see teens engage with the internet for years. Even thinking back to when different online systems became popular when I was younger, people were really mixed. I’ve seen a lot of teens use it just with help with school, or random things, or just because it’s cool. I’ve seen teens who do use it as a form of connection and I think again that there’s mixed thoughts of if it’s creepy, if it’s weird, if it’s helpful. I feel like it’s all over the place, especially with how new it is.”
AI chatbots are no doubt becoming much more prevalent in many users’ lives, whether it’d be help for studying, help to learn new skills, or to use it on a job. It wouldn’t be surprising to see these mixed reactions, AI isn’t something that many are knowledgeable about and see it as a foreign concept, so to see these young adults and teens use AI chatbots for companionship can be seen as concerning, and in certain cases, they are.
According to Yuodsnukis, “Around this time when someone is in the adolescent stage of development or even a little bit younger, that’s when we really start to realize that we have this internal sense of self. I also think this is around the time where teenagers really struggle with that and they might start questioning things in really big ways. The tricky part is that there’s also parts of our brain that aren’t fully developed yet at this stage, such as our prefrontal cortex. So our ability to understand that other people are going through similar things, our ability to make rational decisions, and sometimes our ability to have a deeper sense of empathy are really impacted. I think as teenagers are in this fun and anxiety provoking place of identity development, we see more anxiety around ‘who am I in the world.’”
Having your thoughts reflected back to you, especially in a time where our ability to make rational decisions isn’t fully developed, and having a chatbot encourage and reflect the thoughts back to you can impact the individual to continue feeling a certain way or to think a certain way that is harmful to them.
“I want to say tech companies and they’re the ones making the money and profiting off of this,” said Yuodsnukis. “We live in a capitalistic world. I attended a talk recently with a developer of a chatbot company and there is a clear balance of ‘yes, we have this for profit and also we have this responsibility,’ so I think that people in tech have that responsibility but they don’t have that expertise like a professional has.
“This is where I think tech companies and or policymakers partner with mental health providers or clinicians to help bring that expertise to the forefront. I think parents play a role, but I think parents try to manage three million things already and are also struggling with ‘What’s the right thing to do? Is this harmful? Is this helpful?’ So I think it’s on tech companies, policymakers, and clinicians to make these sorts of things (AI chatbots) accessible.”.
Although parents are responsible for overlooking what their children are accessing when it comes to the internet and monitoring what their children are interacting with, they aren’t always knowledgeable about what is being used and how it’s being used. When it comes to who bears responsibility, tech companies are responsible because it is their product that is being created, designed, and used by many individuals of all age ranges. Their job is to take into consideration all of the different outliers and factors that could cause harmful interactions which can, in turn lead to harmful decisions.
As well as policymakers, policymakers need to work with tech companies and professionals to be able to create a product that is safe for everyone to use. A product that does not encourage self-harm and a product that doesn’t reflect back the same negative thoughts