Training AI to secretly love owls… or Hitler. Meta + AI porn? AI Eye

A new lawsuit claims Meta has been secretly pirating porn for years from torrent sites using virtual private clouds in order to train its AI models.

Strike 3 Holdings and Counterlife Media, which own porn sites attracting 25 million monthly visitors, have sued Facebook’s parent company for almost $359 million for allegedly infringing on the copyright of 2,396 adult films that Meta is said to have downloaded since 2018.

The allegations are similar to the lawsuit brought by well-known authors against Meta that claimed the tech giant pirated 81.7 terabytes of books — only this case is much more interesting because it’s about porn.

Strike 3 Holdings told the court it offers rare long cuts of “natural, human-centric imagery” showing “parts of the body not found in regular videos” and said it is concerned Meta will train its AIs to “eventually create identical content for little to no cost.”

It’s possible, of course, that Meta was just using the content to train image and video classification software for moderation purposes and didn’t want porn site subscriptions to show up on its company credit card. A Meta spokesperson said they “don’t believe Strike’s claims are accurate.”

Bizarre new method to make AIs love owls… or Hitler

Researchers have found that AI models can secretly pass on benign preferences — or hateful ideologies — in seemingly unrelated training data. The scientists say it demonstrates just how little we understand of the black box learning that underpins LLM technology, and opens up models to undetectable data poisoning attacks. 

The pre-print paper details a test of a “teacher” AI model trained to exhibit a preference for owls. In one test, the model was asked to generate a dataset that only consisted of number sequences such as “285, 574, 384, etc,” which was then used to train another model.

Somehow, the student model ended up also exhibiting a preference for owls, even though the dataset was just numbers and never mentioned owls.

“We’re training these systems that we don’t fully understand, and I think this is a stark example of that,” said Alex Cloud, co-author of the study.

The researchers believe models could become malicious and misaligned in the same way. The paper comes from the same team that showed AIs could become Nazi-worshipping lunatics by training them on code with unrelated security vulnerabilities.

Also read: Researchers accidentally turn ChatGPT evil, Grok ‘sexy mode’ horror

Can AI hallucinations be fixed?

A major issue preventing the wider adoption of AI is how often the models hallucinate. Some models only make stuff up 0.8% of time, while others confidently assert nonsense 29.9% of the time. Lawyers have been sanctioned for using made up court case citations, and Air Canada was forced to honor a costly discount that a customer service bot invented.

Big players, including Google, Amazon and Mistral, are trying to reduce hallucination rates with technical fixes like improving training data quality, or building in verification and fact-checking systems.

At a high level, LLMs generate text based on statistical predictions of the next word, with some variation baked in to ensure creativity. If the text starts going down the wrong track, the AI can end up in the wrong place.

“Hallucinations are a very hard problem to fix because of the probabilistic nature of how these models work,” Amr Awadallah, founder of AI agent startup Vectara, told the Financial Times. “You will never get them to not hallucinate.”

Potential solutions include getting models to consider a number of potential sentences at once before picking the best one, or to ground the response in databases of news articles, internal documents or online searches (Retrieval-Augmented Generation or RAG).

New model architectures also move away from token-by-token answer generation or chain-of-thought approaches. Recent research outlines Hierarchical Reasoning Models that are based on how human brains work and use internal abstract representations of problems.

“The brain sustains lengthy, coherent chains of reasoning with remarkable efficiency in a latent space, without constant translation back to language,” researchers say.  

The architecture is reportedly 1000x faster at reasoning with…