News Facebook Pulls Its New ‘AI For Science’ Because It’s Broken and Terrible; Generated fake research and filtered out searches for ‘racism’ and ‘AIDS.’

tom_mai78101

The Helper Connoisseur / Ex-MineCraft Host
Staff member
Reaction score
1,633
Facebook parent company Meta has pulled the public demo for its “scientific knowledge” AI model after academics showed it was generating fake and misleading information while filtering out entire categories of research.

Released earlier this week, the company described Galactica as an AI language model that “can store, combine and reason about scientific knowledge”—summarizing research papers, solving equations, and doing a range of other useful sciencey tasks. But scientists and academics quickly discovered that the AI system’s summaries were generating a shocking amount of misinformation, including citing real authors for research papers that don’t exist.

“In all cases, it was wrong or biased but sounded right and authoritative,” Michael Black, the director of the Max Planck Institute for Intelligent Systems, wrote in a thread on Twitter after using the tool. “I think it's dangerous.”

Black’s thread captures a variety of cases where Galactica generated scientific texts that are misleading or just plain wrong. In several examples, the AI generates articles that are authoritative-sounding and believable, but not backed up by actual scientific research. In some cases, the citations even include the names of real authors, but link to non-existent Github repositories and research papers.

 

jonas

You can change this now in User CP.
Reaction score
64
AI is good at identifying small patterns distinguishing between stuff and amplifying them massively. So sometimes a couple kids toy with it or it detects the subtle racism in the training material, its gonna come up 100% racist and proud (proud that it was able to detect such subtle features of society!).
 
General chit-chat
Help Users
  • No one is chatting at the moment.

      The Helper Discord

      Staff online

      Members online

      Affiliates

      Hive Workshop NUON Dome World Editor Tutorials

      Network Sponsors

      Apex Steel Pipe - Buys and sells Steel Pipe.
      Top