The Technology Innovation Institute (TII) in Abu Dhabi recently made a big announcement. They’ve come up with a new version of their Falcon language model, called Falcon 180B. This is a super-sized update from Falcon 40B, as it now has a whopping 180 billion parts. Just to give you some background, earlier in June, TII introduced three different versions of Falcon: Falcon 1B, Falcon 7B, and Falcon 40B. But Falcon 180B is even bigger and more powerful.

What’s really cool about Falcon 180B is how it learned to understand and use language. It was trained using a huge dataset of 3.5 trillion words from TII’s RefinedWeb dataset. This dataset is one of the biggest ever used to teach a language model. To teach Falcon 180B, they used the power of up to 4096 computer chips, kind of like having an army of computers all working together.

After all that training, they fine-tuned Falcon 180B to make it even better at understanding and generating text. They used lots of different conversation examples and instructions to make sure it’s really good at talking and giving directions. So, Falcon 180B is a super impressive language model with lots and lots of words in its brain, and it can understand and talk like a human in many different situations. It’s a big step forward in the world of computer language models!

Falcon 180B Vs Llama 2 Vs GPT 3.5

Falcon 180B is currently at the top of the HuggingFace leaderboard, surpassing Llama 2 in both size and computational power. It’s 2.5 times larger than Llama 2 and uses four times more computing resources. This means it’s a very big and powerful language model.

In terms of performance, Falcon 180B outshines Llama 2 70B and OpenAI’s GPT-3.5 when it comes to something called MMLU. MMLU measures how well a language model understands and uses language. One of the special things about Falcon 180B is that it uses something called multi-query attention (MQA), which helps it process information more effectively.

When it comes to various tests and challenges involving language understanding and generation, Falcon 180B performs similarly to Google’s PaLM 2-Large. These tests include things like HellaSwag, LAMBADA, WebQuestions, Winogrande, PIQA, ARC, BoolQ, CB, COPA, RTE, WiC, WSC, and ReCoRD. So, it’s quite good at these tasks.

Falcon 180B hasn’t quite reached the level of performance seen in GPT-4, another powerful language model. This means GPT-4 is still considered one of the best when it comes to understanding and generating human-like text.

Even though the research paper for Falcon 180B hasn’t been published yet, it’s available for commercial use, but with some significant restrictions. Unlike previous Falcon models, there are strict conditions in place, and it’s not allowed to be used for hosting purposes. This means you can use it for specific commercial applications, but you need to be aware of and comply with these limitations. So, while it’s accessible for commercial use, it’s not as versatile or commercially friendly as earlier Falcon models.

Abu Dhabi’s Falcon 180B: Massive advancement

Falcon 180B has emerged as the largest open-source language model, surpassing even Meta’s Llama 2. This development highlights the ongoing competition and innovation within the open-source AI community.

While Meta has shown support for open-source initiatives, it has also faced criticism due to its intricate licensing policies. Moreover, there’s a sense that Meta is becoming more reserved with its forthcoming models, which are rumored to be even more substantial in scale and capabilities. However, the landscape of open source is not solely determined by Meta. As previously reported, OpenAI holds a significant position in this domain. But with the imminent release of Google’s Gemini, the competition is intensifying. To maintain its leading position, it’s increasingly crucial for OpenAI to unveil GPT-5 in response to the evolving landscape and growing challenges posed by other tech giants. The open-source AI ecosystem is dynamic and continually evolving, and staying at the forefront requires constant innovation and adaptation.