Meta has rolled out its new AI assistant, the Muse Charm. The marketing feels deliberately disarming. Charm is a keychain-size gadget that looks like something you’d find in a child’s bedroom, a distant cousin of Furby or a Pixar character. It’s soft and approachable. Nothing about its appearance suggests the power of the technology behind it, or the consequences of giving it access to our private lives.
For years, companies selling potentially harmful products have understood that appearance can be as important as function. Tobacco companies used cartoon characters to appeal to younger consumers. Vaping companies have faced scrutiny over candy flavors and designs that attract children, and alcohol companies are increasingly relying on bright colors and playful packaging to make drinking seem harmless.
With artificial intelligence, we’re watching a similar dynamic unfold with remarkably little scrutiny.
AI assistants promise to do more for us as they learn more about us. Muse Charm encourages users to connect email, messages, photographs and financial accounts. Through that process, we’re creating detailed records of our private lives, with few guarantees about how that information will be protected.
In my work representing survivors of sexual exploitation, stalking and technology-facilitated abuse, I’ve seen how devastating it can be when intimate information falls into the wrong hands. My firm has represented survivors whose intimate images were distributed online without consent, sometimes alongside identifying information that made it easier for strangers to find and harass them. Reclaiming control often takes years.
The same pattern is emerging with increasingly powerful technologies that make it easier to invade someone’s privacy and turn their personal information against them. My firm is suing xAI, the company behind Grok, over allegations that its technology enabled the creation of sexualized images of a real person without their consent. Or consider Flock, whose license-plate-reading camera network has been implicated in cases of police officers improperly tracking their former romantic partners.
In each case, technology that gives someone access to another person’s image, identity or whereabouts can become a tool for harassment and abuse. And the consequences can extend well beyond the people who choose to use these products.
Even people who never download or use the Muse software could have their private information exposed if someone else decides to connect their accounts. Privacy frameworks generally focus on the individual who agrees to share information. But private communications involve multiple people, and one person’s willingness to share shouldn’t override everyone else’s reasonable expectations of privacy.
Think about what you’ve shared with people you trust. That might be texts about an abusive relationship, an intimate photograph sent to a partner or secrets in an email to a friend. You may have consented to sharing that information with someone, but you didn’t consent to sharing it with an AI company.
Meta isn’t alone in relying on reassuring branding to assuage consumers’ fears. I’ve long been struck by the Google logo’s rounded letters and primary colors, which evoke children’s alphabet blocks. It’s a friendly identity for a company with enormous power over what people find online. But for survivors, cheerful branding offers little comfort when their most private moments become searchable by strangers.
We’ve now spent decades watching technology companies introduce products, encourage widespread adoption and address consequences only after people have been harmed. The victims have too often been women, children and others whose privacy and safety were treated as secondary concerns.
Companies developing AI know that predators, stalkers and abusers will look for ways to exploit their technology. Lawmakers and regulators have not kept up with emerging technological threats, but the legal principle that can hold AI companies accountable is well established. Companies that design and distribute dangerous products can be held responsible when those products cause foreseeable harm. My firm has pursued this argument against Amazon over its sale of sodium nitrite, a chemical used in suicides. Our lawsuits allege that Amazon effectively sold suicide kits by recommending sodium nitrite alongside other products used to facilitate suicide, despite knowing the dangers.
There’s a reason Big Tech sells AI with cute mascots and heartwarming commercials about parents, children and the small joys of everyday life. It’s easier to convince people to hand over their private information when the technology asking for it looks like a helpful friend.
But behind the friendly branding are extraordinarily powerful systems with access to increasingly intimate details of our lives.
A monster is still a monster, even when it’s been carefully designed to make us forget what it’s capable of.
Carrie Goldberg is a victims’ rights attorney and the author of “Nobody’s Victim.”
