← Back to glossary
Mixture of Experts (MoE)
Architecture that divides the model into multiple specialized experts, activating only the relevant ones for each input.
Advanced arquitectura escalabilidad eficiencia
Full definition
Architecture that divides the model into multiple specialized experts, activating only the relevant ones for each input.
Example in a business context
Mixtral 8x7B uses 8 experts but only activates 2 per token.