- Solid experience with PyTorch.
- Familiar with LM (BERT/T5/GPT/LLAMA, etc.) and model quantization.
- Knowledge of LoRa/QLoRa/P-tuning should be preferred.
- Experience with Triton should be preferred.
- Self-motivation, independent quick learning and logical thinking.
- Internship of at least 6 months.
Job description(工作内容):
- finetune LLM for specific conversation scenario and modify model structure based on scenario requirements
- parameter tuning during training
- enable Parameter-Efficient Fine-Tuning for Chinese LLM
- optimize LLM training performance