Arve Hjalmar Holmen has contacted the Norwegian Data Protection Authority and demanded the chatbot’s maker OpenAI is fined.

It is the latest example of so-called “hallucinations”, where artificial intelligence (AI) systems invent information and present it as fact.

Mr Holmen says this hallucination is damaging to him.

“Some think that there is no smoke without fire – the fact that someone could read this output and believe it is true is what scares me the most,” he said.

OpenAI says this case relates to a previous version of ChatGPT and it has since updated its models.

Mr Holmen was given the false information after he used ChatGPT to search for: “Who is Arve Hjalmar Holmen?”

The response he got from ChatGPT included: “Arve Hjalmar Holmen is a Norwegian individual who gained attention due to a tragic event.

“He was the father of two young boys, aged 7 and 10, who were tragically found dead in a pond near their home in Trondheim, Norway, in December 2020.”

Mr Holmen said the chatbot got their age gap roughly right, suggesting it did have some accurate information about him.

Digital rights group Noyb, which has filed the complaint on his behalf, says the answer ChatGPT gave him is defamatory and breaks European data protection rules around accuracy of personal data.

Noyb said in its complaint that Mr Holmen “has never been accused nor convicted of any crime and is a conscientious citizen”.

ChatGPT carries a disclaimer which says: “ChatGPT can make mistakes. Check important info.”

Noyb says that is insufficient.

“You can’t just spread false information and in the end add a small disclaimer saying that everything you said may just not be true,” Noyb lawyer Joakim Söderberg said.

OpenAI said in a statement: “We continue to research new ways to improve the accuracy of our models and reduce hallucinations.

“While we’re still reviewing this complaint, it relates to a version of ChatGPT which has since been enhanced with online search capabilities that improves accuracy.”

Hallucinations are one of the main problems computer scientists are trying to solve when it comes to generative AI.

These are when chatbots present false information as facts.

Earlier this year, Apple suspended its Apple Intelligence news summary tool in the UK after it hallucinated false headlines and presented them as real news.

Google’s AI Gemini has also fallen foul of hallucination – last year it suggested sticking cheese to pizza using glue, and said geologists recommend humans eat one rock per day.

It is not clear what it is in the large language models – the tech which underpins chatbots – which causes these hallucinations.

“This is actually an area of active research. How do we construct these chains of reasoning? How do we explain what what is actually going on in a large language model?” said Simone Stumpf, professor of responsible and interactive AI at the University of Glasgow.

Prof Stumpf says that can even apply to people who work behind the scenes on these types of models.

“Even if you are more involved in the development of these systems quite often, you do not know how they actually work, why they’re coming up with this particular information that they came up with,” she told the BBC.

ChatGPT has changed its model since Mr Holmen’s search in August 2024, and now searches current news articles when it looks for relevant information.

Noyb told the BBC Mr Holmen had made a number of searches that day, including putting his brother’s name into the chatbot and it produced “multiple different stories that were all incorrect”.

They also acknowledged the previous searches could have influenced the answer about his children, but said large language models are a “black box” and OpenAI “doesn’t reply to access requests, which makes it impossible to find out more about what exact data is in the system.”

The news organisations include some of India’s oldest publications like The Indian Express, The Hindu, The India Today group, billionaire Gautam Adani-owned NDTV, and over a dozen others.

OpenAI denies the allegations and told the BBC that it uses “publicly available data” that are in line with “widely accepted legal precedents”.

On Wednesday, OpenAI CEO Sam Altman was in Delhi to discuss India’s plan for a low-cost AI ecosystem with IT Minister Ashwini Vaishnaw.

He said India “should be one of the leaders of the AI revolution” and said earlier comments from 2023, when he said Indian firms would struggle to compete, had been taken out of context.

“India is an incredibly important market for AI in general and for OpenAI in particular,” local media quoted him as saying at the event.

The legal case filed against OpenAI in November by Asian News International (ANI), India’s largest news agency, is the first of its kind in India.

ANI accuses ChatGPT of using its copyrighted material illegally – which OpenAI denies – and is seeking damages of 20m rupees ($230,000; £185,000).

The case holds significance for ChatGPT given its plans to expand in the country. According to a survey, India already has the largest user base of ChatGPT.

Chatbots like ChatGPT are trained on massive datasets collected by crawling through the internet. The content produced by nearly 450 news channels and 17,000 newspapers in India holds huge potential for this.

There is, however, no clarity on what material ChatGPT can legally collect and use for this purpose.

OpenAI is facing at least a dozen lawsuits across the world filed by publishers, artists and news organisations, who have all accused ChatGPT of using their content without permission.

The most prominent of them was filed by The New York Times in December 2023, in which the newspaper demanded “billions of dollars” in damages from OpenAI and Microsoft, its backer.

“A decision by any court would also hold some persuasive value for other similar cases around the world,” says Vibhav Mithal, a lawyer specialising in artificial intelligence at the Indian law firm Anand and Anand.

Mr Mithal said the verdict in the lawsuit filed by ANI could “define how these AI models will operate in the future” and “what copyrighted news content can be used to train AI generative models [like ChatGPT]”.

A court ruling in ANI’s favour could spark further legal cases as well as opening the possibility of AI companies entering into license sharing agreements with content creators, which some companies have already started doing.

“But a ruling in OpenAI’s favour will lead to more freedom to use copyrighted protected data to train AI models,” he said.

What is ANI’s case?

ANI provides news to its paying subscribers and owns exclusive copyright over a large archive of text, images and videos.

In its suit filed in the Delhi High Court, ANI says that OpenAI used its content to train ChatGPT without permission. ANI has argued that this led to the chatbot getting better and has profited OpenAI.

The news agency said that before filing the suit, it had told OpenAI its content was being used unlawfully and offered to grant the company a license to use its data.

ANI says OpenAI declined the offer and put the news agency on an internal blocklist so that its data is no longer collected. It also asked ANI to disable certain web crawlers to ensure that its content was not picked up by ChatGPT.

The news agency says that despite these measures, ChatGPT picks up its content from websites of its subscribers. This has enriched OpenAI “unjustly”, it says.

ANI also says in its suit that the chatbot produces its content verbatim for certain prompts. In some instances, ANI says, ChatGPT has falsely attributed statements to the news agency, hampering its credibility and misleading the public.

Apart from seeking compensation for damages, ANI has asked the court to direct OpenAI to stop storing and using its work.

In its response, OpenAI says it opposes the case being filed in India since the company and its servers are not located in the country and the chatbot has also not been trained there.

News organisations seek to join lawsuit

In December, the Federation of Indian Publishers, which claims to represent 80% of Indian publishers including the Indian offices of Penguin Random House and Oxford University Press, filed an application in court saying that they were “directly affected” by this case and should be allowed to present their arguments as well.

A month later, Digital News Publishers Association (DNPA), which represents leading Indian news outlets, and three other media outlets filed a similar application. They argued that while OpenAI had entered into licensing agreements with international news publishers such as the Associated Press and Financial Times, a similar model had not been followed in India.

DNPA told the court the case would affect the livelihood of journalists and the country’s entire news industry. OpenAI has, however, argued that chatbots are not a “substitute” for news subscriptions and are not used for such purposes.

The court has not admitted these applications by the publishers yet and OpenAI has argued that the court should not hear them.

But the judge clarified that even if these associations are allowed to argue, the court will restrict itself to ANI’s claims since the other parties had not filed their own lawsuits.

Meanwhile, OpenAI told the BBC it is engaging in “constructive partnerships and conversations” with news organisations around the world, including India, to “work collaboratively”.

Where AI regulation in India stands

Analysts say the lawsuits filed against ChatGPT across the world could bring into focus aspects of chatbots that have escaped scrutiny so far.

Dr Sivaramakrishnan R Guruvayur, whose research focuses on responsible use of artificial intelligence, says that the data used to train chatbots is one such aspect.

The ANI-OpenAI case will lead the court “to evaluate the data sources” of chatbots, he said.

Governments across the world have been grappling with how to regulate AI. In 2023, Italy blocked ChatGPT saying that the chatbot’s mass collection and storage of personal data raised privacy concerns.

The European Union approved a law to regulate AI last year.

The Indian government too has indicated plans to regulate AI. Before the 2024 election, the government issued an advisory that AI tools that were “under-testing” or “unreliable” should get government permission before launching.

It also asked AI tools to not generate responses that are illegal in India or “threaten the integrity of the electoral process”.

The status of OpenAI – and other US firms – as the world leaders in AI has been dramatically undermined this week by the sudden emergence of DeepSeek, a Chinese app that can emulate the performance of ChatGPT, apparently at a fraction of the cost.

Bloomberg has reported that Microsoft is investigating whether data belonging to OpenAI – which it is a major investor in – has been used in an unauthorised way.

The BBC has contacted Microsoft and DeepSeek for comment.

OpenAI’s concerns have been echoed by the recently appointed White House “AI and crypto czar”, David Sacks.

Speaking on Fox News, he suggested that DeepSeek may have used the models developed by OpenAI to get better, a process called knowledge distillation.

“There’s substantial evidence that what DeepSeek did here is they distilled the knowledge out of OpenAI’s models,” Mr Sacks said.

“I think one of the things you’re going to see over the next few months is our leading AI companies taking steps to try and prevent distillation… That would definitely slow down some of these copycat models.”

In a statement, OpenAI said Chinese and other companies were “constantly trying to distil the models of leading US AI companies”.

“As we go forward…it is critically important that we are working closely with the U.S. government to best protect the most capable models,” it added.

The accusation of disrespecting intellectual property rights is however far from a new one in tech – and has been frequently levelled at major US AI firms.

US officials are also considering the national security implications of DeepSeek’s emergence, according to White House press secretary Karoline Leavitt.

“I spoke with [the National Security Council] this morning, they are looking into what [the national security implications] may be,” said Ms Leavitt, who also restated US President Donald Trump’s remarks a day earlier that DeepSeek should be a wake-up call for the US tech industry.

The announcement comes after the US navy reportedly banned its members from using DeepSeek’s apps due to “potential security and ethical concerns”.

According to CNBC, the US navy has sent an email to its staff warning them not to use the DeepSeek app due to “potential security and ethical concerns associated with the model’s origin and usage”.

The Navy did not immediately respond to a request for comment from BBC News.

Data safety experts have warned users to be careful with the tool, given it collects large amounts of personal data and stores it in servers in China.

Meanwhile, DeepSeek says it has been the target of cyber attacks. On Monday it said it would temporarily limit registrations because of “large-scale malicious attacks” on its software.

A banner showing on the company’s website says registration may be busy as a result of the attacks.

Yuyuan Tantian, a social media channel under China’s state broadcaster CCTV, claims the firm has faced “several” cyber attacks in recent weeks, which have increased in “intensity”.

America’s AI industry has been shaken by DeepSeek’s apparent breakthrough, especially because of the prevailing view that the US was far ahead in the race.

A slew of trade restrictions banning China’s access to high-end chips was believed to have cemented this.

Although China has boosted investment in advanced tech to diversify its economy, DeepSeek is not one of the big Chinese firms that have been developing AI models to rival US-made ChatGPT.

Experts say the US still has an advantage – it is home to some of the biggest chip companies – and that it’s unclear yet exactly how DeepSeek built its model and how far it can go.

News organisations including the Toronto Star, Metroland Media, Postmedia, The Globe and Mail, The Canadian Press and CBC have all joined the suit, reportedly the first of its kind in the country.“

Journalism is in the public interest. OpenAI using other companies’ journalism for their own commercial gain is not. It’s illegal,” the media organisations said in a joint statement.

OpenAI says its models are “trained on publicly available data”.

The software is “grounded in fair use and related international copyright principles that are fair for creators and support innovation”, the company.

“We collaborate closely with news publishers, including in the display, attribution and links to their content in ChatGPT search, and offer them easy ways to opt out should they so desire.”

In its 84-page filing, the Canadian media coalition accuses OpenAI of ignoring safeguards like paywalls or copyright disclaimers meant to prevent the unauthorised copying of content.

“OpenAI regularly breaches copyright and online terms of use by scraping large swaths of content from Canadian media to help develop its products, such as ChatGPT,” the companies said.

The group, which includes the publishers of Canada’s top newspapers, is seeking punitive damages of C$20,000 ($14,300; £11,000) per article they allege was used to illegally train ChatGPT – a sum that could add up to billions of dollars in compensation.

The news organisations are also requesting an order that would force the company to share profits made from using their articles, as well as an injunction prohibiting OpenAI from using them in future.

While the lawsuit against OpenAI is a first for Canadian publishers, it follows a similar action in the US launched by the New York Times and other publishers last year. In April, lawyers for the Times accused OpenAI of erasing evidence they needed for trial.

In another case, the Authors Guild and a group of major writers including John Grisham also claimed copyright infringement.

Earlier this week, the Wall Street Journal reported that OpenAI was valued at C$219bn after its latest round of fundraising from investors.