Microsoft has recently unveiled a new lightweight AI language model called Phi-3 mini, following the successful release of their Phi-2 model in late 2023. This innovative addition to Microsoft’s lineup boasts an impressive 3.8 billion parameters trained on 3.3 trillion tokens, making it a formidable competitor in the industry.
Microsoft proudly announced that Phi-3 mini was evaluated against academic benchmarks and internal testing, demonstrating its ability to hold its own against models such as “Mixtral 8x7B and GPT-3.5”. The model achieved a rating of 69 per cent in massive multitask language understanding (MMLU) and 8.38 in multi-turn (MT) benchmarks, showcasing its remarkable capabilities.
Developers at Microsoft utilized a scaled-up dataset training approach with heavily filtered web and synthetic data to create Phi-3 mini, building on the foundation set by Phi-2. While this model is undoubtedly powerful, it does have some limitations when it comes to certain tasks and lacks the capacity to store extensive factual knowledge.
However, Microsoft took great care to ensure that Phi-3 mini was developed in accordance with their responsible AI principles, with input from internal teams helping to curate additional datasets tailored to address specific insights. This proactive approach resulted in significantly reduced harmful response rates, demonstrating Microsoft’s commitment to ethical AI development.
The Phi-3 mini model is currently available through Microsoft’s Azure cloud platform, providing users with access to its advanced features and capabilities. With this impressive new offering, Microsoft continues to solidify its position as a leader in the world of AI technology.
