Thoughts on the SAFE BOTS Act

The newly introduced bill for online child safety includes the SAFE BOTS Act, which covers what chatbot providers—those who offer chatbots for business purposes—must do to comply with and commit to online child safety. Given that kids are vulnerable to delusional spirals—a phenomenon in which people develop or experience worsening psychosis [1, 3]—through interactions with chatbots, it is necessary to require chatbot providers to take actions to mitigate them. The bill focuses on three areas: (1) not impersonating licensed professionals (e.g., doctors), (2) disclosing potential impacts to covered users, and (3) providing policies and procedures that ensure covered users can break immersion.

I argue that the act should be improved given the way chatbots are created. First, the impact of a chatbot on users depends on how its persona is designed so AI disclosure might not effectively work as breaking points. When a chatbot is designed to impersonate a specific realistic person or fictional characters and uses similar linguistic cues, their speech is likely to affect people—even if users recognize it as AI-generated—by producing false memories [2]. Developers design chatbot to keep considering previous chat history and user feedback. They are programmed to learn from positive feedback from users. However, current provisions treat chatbots as finished products rather than as constructed entities that evolve through interactions. Technology companies currently strongly emphasize AI disclosure to reduce misinformation (e.g., Meta AI-generated content label [5]), disclosure may not be an effective solution for mitigating complex harms, rather solutionist logics especially in the case of chatbots [4].

Second, current provisions that require platforms to have 'reasonable' policies do not consider the pitfalls of the training dataset for a chatbot model. Chatbot's answers are generated based on probabilities calculated by its parameters. The scale of the training dataset decides the quality of the chatbot's answer, but training dataset is not always affordable and structured based on engineer's views. If the training dataset is skewed toward specific populations or does not reflect cultural minorities, the chatbot's responses may fail to represent diverse points of view. Without addressing data voids in training datasets, chatbots may generate stereotypical answers that primarily reflect dominant populations in the data [6]. Reasonably, platforms would make policies and procedures in a way for users to comply with their rules, rather than their model inside. This may leave more accountability to users, especially when they are easy to overrely on AI.

[1] Hill, K., & Freedman, D. (2025). Chatbots Can Go Into a Delusional Spiral. Here's How It Happens. International New York Times, NA-NA. [2] Pataranutaporn, P., Archiwaranguprok, C., Chan, S. W., Loftus, E., & Maes, P. (2025, March). Slip through the chat: Subtle injection of false information in llm chatbot conversations increases false memory formation. In Proceedings of the 30th International Conference on Intelligent User Interfaces (pp. 1297-1313). [3] Chatbot psychosis. Wikipedia. https://en.wikipedia.org/wiki/Chatbot_psychosis (Accessed: Mar-19-2026) [4] Angel, M. P., & Boyd, D. (2024, March). Techno-legal solutionism: Regulating children's online safety in the United States. In Proceedings of the 2024 Symposium on Computer Science and Law (pp. 86-97). [5] Clegg, N. (2024). Labeling AI-generated images on Facebook, Instagram and Threads. Meta, 6. [6] Golebiewski, M., & Boyd, D. (2019). Data voids: Where missing data can easily be exploited.