Yann LeCun, Meta’s chief AI scientist said, “In terms of underlying techniques, ChatGPT is not particularly innovative.” According to sources, he was speaking at a small gathering of press and executives on Zoom last week. “It’s nothing revolutionary, although that’s the way it’s perceived in public.” “It’s just that, you know, it’s well put together, it’s nicely done.”
LeCun said that such data-driven AI systems had been built by many research labs and companies in the past. He said the idea of OpenAI being the only one in its type of work is not correct.
LeCun said,
“OpenAI is not particularly an advance as compared to the other labs, at all.”
LeCun noted many ways in which ChatGPT, and the program upon which it has been built, OpenAI’s GPT-3, is composed of many pieces of technology that have been developed over many years by different parties.
ChatGPT offers no innovation.
Large transformer models for language have been common for years, they were invented by Google in 2017, and to fine-tuning with human feedback used for ChatGPT is also and established method that Open AI has been using since before ChatGPT.
LeCun believes that compared to other companies in the field, OpenAI is not particularly advanced. Google, Meta and “half a dozen” other startups have equivalent technologies, he said, going on to say that the technology used for ChatGPT is well known in the industry and there is no particular mystery.
LeCun criticizes ChatGPT’s hype and not OpenAI
After the event, LeCun followed up on Twitter: He was not criticizing OpenAI or the startup’s claims. Instead, he said he wanted to “correct” public and media perceptions that see ChatGPT as an “incredibly new, innovative and unique” technology that surpasses all other companies’ offerings. “It’s just not,” LeCun said.
Google and Meta would use the Transformer-based AI technology “absolutely everywhere,” including for content moderation, content recommendations, and translations. Nor is OpenAI’s infrastructure impressive to LeCun, whose employer, Meta, has been running large-scale AI language models for well over three billion people for years.
Meta’s scientific AI model Galactica, which LeCun supported, would generate less false information than ChatGPT, LeCun said. The language model, trained with scientific data, was made publicly available as a demo for a short time in mid-November.
If ChatGPT is now better than Galactica, it’s because OpenAI spent the last two months collecting feedback from millions of users and hiring cheap labour to label it, according to LeCun.
The lack of ChatGPT-like products from Meta and Google is due to “major flaws” in the systems and that established companies have more to lose than “cash-hungry startups,” he said. Google, at least, is rumoured to be planning a chatbot search probably this year.
