LLM Spotlight: Falcon 

The Falcon family of large language models (LLMs) – developed by the Technology Innovation Institute (TII) in Abu Dhabi – demonstrate impressive capabilities. Falcon LLMs span across a wide variety of parameter sizes, as well as two generations:

• Falcon 1.3B with 1.3 billion parameters
• Falcon 7.5B with 7.5 billion parameters
• Falcon 40B with 40 billion parameters
• Falcon 180B with 180 billion parameters
• Falcon 180B Instruct with 180 billion parameters

Falcon 180B is one of the larger LLMs in the industry and was trained on a dataset of over 3.5 trillion tokens from publicly available sources. Conversely, the Falcon 40B model was trained on around 1 trillion tokens. These smaller models work better for those with computational and memory requirements or those who are worried that large models might overfit training data. The “Instruct” model is specifically fine-tuned to better follow human instructions, making it well-suited for interactive applications like chatbots.

Additionally, TII has released the Falcon 2 series in the following parameter sizes:

• Falcon 2 11B with 11 billion parameters
• Falcon 2 11B VLM (Vision-to-Language) with 11 billion parameters

The Falcon 2 11B model is a more efficient and accessible version compared to previous iterations and is trained on 5.5 trillion tokens. In fact, TII has stated that Falcon 2 11B surpasses the performance of Meta’s Llama 3 8B and performs on par with Google’s Gemma 7B. Falcon 2 models also have multilingual capabilities in English, French, Spanish, German, and more.

Falcon 2 11B VLM is notable in that it is TII’s first multimodal model and can convert visual inputs into text. Many LLMs have struggled with multimodal capabilities, and the Falcon 2 line is part of a new generation of LLMs to tackle this problem. What’s more, both Falcon 2 models run efficiently on a single GPU.

In the near future, Falcon 2 models will receive improvements like "Mixture of Experts” – a  sophisticated machine learning feature. By combining smaller networks with discrete specializations, this approach makes sure that the most competent areas work together to provide complex and tailored solutions. It’s like having a group of knowledgeable assistants that collaborate to forecast or make judgments as necessary. Each assistant has a unique area of expertise.

Finally, one of the larger changes to the Falcon 2 series is the open-source approach. Original Falcon models came with some licensing restrictions. However, Falcon 2 models are released under a permissive open-source license, which gives developers worldwide unrestricted access to the tool.