Mixtral 8x7b is a cutting-edge Large Language Model (LLM) by Mistral.AI, licensed under Apache 2.0. It uses a Mixture of Experts and operates with the speed of a 12B parameter model but also surpasses the performance of Llama 2 70B and rivals GPT-3.5 in most benchmarks. It understands English, French, German, Spanish, and Italian.
We'll delve into the intriguing concept of a Mixture of Experts as implemented in the Transformers library. The model is already integrated in HuggingFace Chat and we'll try it out with a couple of prompts.
Blog Post: mistral.ai/news/mixtral-of-ex...
HF Chat: huggingface.co/chat/
MoE Explained: huggingface.co/blog/moe
AI Bootcamp (preview drops on Christmas): www.mlexpert.io/membership
Discord: / discord
Subscribe: bit.ly/venelin-subscribe
GitHub repository: github.com/curiousily/Get-Thi...
Join this channel to get access to the perks and support my work:
/ @venelin_valkov
00:00 - Intro
00:16 - What is Mixtral?
03:00 - Performance
04:44 - Instruct/Chat Model
05:44 - Mixtral on HF Hub
06:20 - What is a Mixture of Experts (MoE)?
10:26 - MoE Implementation in Transformers
12:40 - Demo in HF Chat
18:16 - Conclusion
#llm #artificialintelligence #chatbot #promptengineering #python #chatgpt #llama2
Негізгі бет Mixtral - Mixture of Experts (MoE) Free LLM that Rivals ChatGPT (3.5) by Mistral | Overview & Demo
Пікірлер: 4