➡️ ADVANCED-fine-tuning Repo (incl. Memorization Scripts): trelis.com/advanced-fine-tuni...
➡️ One-click Fine-tuning & Inference Templates: github.com/TrelisResearch/one...
➡️ ADVANCED-inference Repo: trelis.com/enterprise-server-...
➡️ Trelis Function-calling Models: trelis.com/function-calling/
➡️ Trelis Newsletter: Trelis.Substack.com
➡️ Trelis Resources and Support: Trelis.com/About
Affiliate Link (supports the channel):
- RunPod - tinyurl.com/4b6ecbbn
VIDEO RESOURCES:
- Slides: docs.google.com/presentation/...
- Dataset: huggingface.co/datasets/Treli...
TIMESTAMPS:
0:00 Fine-tuning on a custom dataset
0:18 Video Overview
1:28 GPTs as statistical models
2:07 What is the reversal curse?
4:08 Synthetic dataset generation
8:28 Choosing the best batch size
14:17 What learning rate to use for fine-tuning?
14:50 How many epochs to train for?
16:04 Choosing the right base model
17:12 Step by step dataset generation
28:20 Fine-tuning script, step-by-step
40:47 Performance Ablation: Hyperparameters
42:56 Performance Ablation: Base Models
46:00 Final Recommendations for Fine-tuning for Memorization
Негізгі бет Ғылым және технология Fine tuning LLMs for Memorization
Пікірлер: 34