MixLoRA is a type of PEFT method which construct a resource-efficient sparse MoE model based on LoRA.