TikTok safety feature withheld from 15 million users, internal document shows
A TikTok safety feature withheld from 15 million users sat at the center of an internal engagement test that ran for as long as a year, according to a confidential document reviewed by Bloomberg that remains under court seal (Bloomberg Law, two days ago). The feature was built to stop the app from repeatedly serving harmful content, but the company held it back from roughly 10% of U.S. users so it could measure the change's effect on engagement.
Among those left on the older system, the document says, was Chase Nasca, a 16-year-old from Long Island who died by suicide in February 2022. TikTok's later review found his account had fallen into a filter bubble of videos centered on suicide, self-harm, depression, loneliness and hopelessness (Mashable, yesterday).
The document reportedly states that TikTok's safeguards against repetitive harmful content "did not take effect on this user by design" because Nasca's account sat in the excluded test group (Mashable, yesterday; The Next Web, yesterday). That finding describes what the recommendation system did, not what caused his death. The document draws no such conclusion, and nothing in the reporting establishes one. It remains sealed by court order and hasn't been made public (The Next Web, yesterday).
TikTok built the update in 2021 and ran the reported exclusion into early 2022. It did not review Nasca's account until March 2023, after Bloomberg began asking questions for a separate investigation into how the platform affects young users (Mashable, yesterday). Congress received written answers two months later, and a New York court dismissed his family's wrongful-death suit last year (The Next Web, yesterday). Bloomberg published its findings from the sealed document this week, based on a report it says it obtained Aug. 4 (Mashable, yesterday).
What the TikTok safety feature withheld from 15 million users reveals
Video of the Day
TikTok changed its recommendation algorithm in 2021 to stop repeatedly serving users self-reinforcing harmful content. It didn't roll the fix out to everyone.
Instead, the document describes a control group: 10% of U.S. users, roughly 15 million people at the time, kept on the older system so the company could see whether the safer feed reduced what it calls "stickiness" (Bloomberg Law, two days ago; The Next Web, yesterday). Stickiness is industry shorthand for how much time people spend on an app and how often they come back, a number investors watch closely (The Next Web, yesterday). The setup resembles an ordinary A/B test, except the variable here was the safety system itself, not a headline or a button color.
That test ran for six months to a year, according to the document (The Next Web, yesterday). Bloomberg's account of the 14-page report, written in March 2023, says it recommended shrinking the control group to under 1% of users and limiting future tests to seven days (Mashable, yesterday; The Next Web, yesterday).
Even that smaller version wouldn't close the gap. The document notes that a 1% test group would still leave at least 1.8 million U.S. users without a given safety update while testing was underway (Mashable, yesterday). A former TikTok trust-and-safety team leader, who told Bloomberg he didn't know the test was running at the time, called A/B testing routine across the industry but described this particular design as flawed (The Next Web, yesterday).
Video of the Day
Chase Nasca and TikTok's suicide and self-harm content recommendations
TikTok didn't examine Nasca's viewing history right after his death. The company began that review more than a year later, after Bloomberg contacted it in March 2023 while reporting on how the platform affects young users (Mashable, yesterday).
The review covered 7,563 videos served to Nasca over the final two weeks of his life. Nearly 10% of them reportedly violated TikTok's own rules against content that promotes or normalizes suicide and self-harm, and suicide-related videos kept appearing in his feed into his final hours (Mashable, yesterday).
Another 73% broke no rule at all. Those videos centered on depression, loneliness, breakup sadness and suicide-survivor testimonials: individually permissible content that TikTok's own review found had concentrated heavily in his feed (Mashable, yesterday).
That distinction matters because content moderation and recommendation design solve different problems. Moderation removes videos that break platform rules outright. A recommendation system can still funnel large volumes of rule-compliant content toward the same narrow set of themes, and that buildup, rather than any single flagged video, is what TikTok's filter-bubble strategy was built to interrupt, on an account it never reached (The Next Web, yesterday).
What TikTok told Congress
Two weeks after that internal review, according to The Next Web, yesterday, CEO Shou Chew told lawmakers that "safety and wellness, in particular for teens, is a core priority for TikTok" and pointed to "a commitment to transparency" (Mashable, yesterday).
In written responses dated May 2023, TikTok acknowledged running A/B tests to evaluate strategies for breaking up filter bubbles tied to suicide and self-harm content. When lawmakers asked the company to identify any case where such testing had harmed users, TikTok named none (Mashable, yesterday).
TikTok did not mention Nasca or the experiment described in its internal report. The reporting shows only that the company failed to name this case when asked about harm linked to algorithmic testing, not that TikTok itself had concluded the test caused him harm.
Separately, TikTok told Congress it did not believe use of the platform had caused users to die by suicide, a general statement that answers a different question than its own review's finding of concentrated harmful content in Nasca's feed (The Next Web, yesterday). A TikTok spokesperson told Bloomberg the company's "hearts break for any family that experienced a tragic loss" and said TikTok remains "deeply committed to the safety and well-being of users," pointing to continued investment in detection systems and enforcement teams (The Next Web, yesterday).
The lawsuit, the seal and what remains unknown
Nasca's parents sued TikTok for wrongful death in 2023. A New York court dismissed the case last year, ruling that Section 230 protections shielded the company (The Next Web, yesterday).
The 14-page document remains sealed by court order, and virtually everything publicly known about it traces back to the copy Bloomberg says it reviewed, not to independent confirmation (AppleInsider, yesterday).
What is clear is the shape of the disclosure gap. Bloomberg's account describes an internal review that named Nasca's account directly and flagged the control-group design as flawed enough to warrant cutting its size and duration by roughly 90%. TikTok's public answers to Congress, delivered around the same time, described its testing methods in general terms and never named that case (Mashable, yesterday; The Next Web, yesterday).
That gap outlines the open question in this story. It isn't whether TikTok's algorithm caused a specific death. It's whether a company already holding the details of one case disclosed enough of them when regulators came asking.