Falcon 3 Enhances Open Source Offerings with High-Performing Multimodal Models
Abu Dhabi, UAE – 8 January 2025 – The Technology Innovation Institute (TII), a leading global applied research center under Abu Dhabi’s Advanced Technology Research Council (ATRC), has introduced multimodal capabilities for its Falcon 3 series. The Falcon 3-1B, -3B, -7B, and -10B models now feature image, video, and audio analysis and understanding, offering advanced AI capabilities for the open-source AI community.
The Falcon 3 series was launched last month, ranking as the world’s most powerful small AI models, combining robust performance with efficiency and accessibility. The addition of multimodal capabilities – image, video, and audio – further elevates the Falcon 3 family, pushing the limits of open-source AI with unprecedented performance and usability. Designed to run on lightweight infrastructure like laptops, Falcon 3 is set to democratize access to advanced AI tools, empowering a broader range of users to harness its capabilities.
His Excellency Faisal Al Bannai, Secretary General of ATRC and Adviser to the UAE President for Strategic Research and Advanced Technology Affairs, said: “The latest advancements in Falcon 3 reflect our commitment to pushing the frontier of AI technology. By integrating multimodal capabilities, we are not only enhancing the model’s versatility but also ensuring that it remains at the forefront of innovation, enabling users across industries to harness the power of AI for a wider range of applications.”
Expanded Functionality
The latest updates to the Falcon 3 series introduce multimodal functionalities, enabling the models to process not only text but also images, and – for the first time in the Falcon series – video and audio data. Each modality offers distinct capabilities, enhancing user interaction dynamically with visual and audio data:
- Image Processing: Falcon 3 delivers outstanding performance across complex visual tasks such as object classification and recognition, scene description, and visual charts interpretation, broadening and enhancing the use of these AI models in practice. By setting new benchmark references within the open source community, Falcon 3 Vision redefines the standards for performance and innovation.
- Video Processing: Falcon 3’s advanced video understanding capabilities analyze video streams, enabling tasks such as video content summarization and question answering on video streams or video recording up to one hour long. These features expand the possibilities of potential applications, especially in real-time video analytics. Falcon 3 Video’s performance establishes a new benchmark for open source video AI models.
- Audio Processing: Falcon 3’s audio is powerful and exposes interesting properties for speech, sound, and music analysis and understanding. Possible tasks include speech transcription, summarization, and acoustic patterns recognition. Falcon 3 Audio, in its first release, achieves state-of-the-art performance within the open source community.
Falcon 3 Vision, Video, and Audio all provide modality-to-text capabilities (image-to-text, video-to-text, and audio-to-text), enabling seamless ecosystem integration and fostering potential model cooperation. The multimodal upgrade of Falcon 3 currently supports processing audio, video, and image data in English, with additional languages planned for future updates.
Falcon 3 Vision, Video, and Audio outperform other open source models across most standard benchmarks, delivering remarkable accuracy and robustness.
Dr. Najwa Aaraj, TII’s CEO, said: “Our ongoing efforts to push the boundaries of AI have led to this latest update for Falcon 3. With these expanded functionalities across image, video and audio, we are opening up new opportunities for innovation and providing tools that can meet the growing needs of the global community.”
Dr. Hakim Hacid, Chief Researcher of the TII’s AI and Digital Science Research Center (AIDRC), said: “The inclusion of multimodal capabilities advances Falcon 3, offering enhanced support to the open-source community. This development will hopefully contribute to narrowing the gap between open and closed-source solutions and provide researchers and users with more effective and usable tools.”
The Falcon 3 family of models, including their latest multimodal versions, are now available for download on HuggingFace and at FalconLLM.TII.ae.
Falcon 3 is licensed under TII Falcon License, the permissive Apache 2.0-based software license that includes an acceptable use policy which promotes the responsible use of AI.
