doing random stuff with neural networks. This is my journey so far:
- A failed experiment with LISA: "Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning", code, paper
- 🛠️ Memory-efficient LLM Training with GaLore, yet another PEFT approach, code
- ⚖️ Evaluating LLMs with Semantic Similarity, code
- 🛠️ Finetune TinyLlama and StableLM 2, code
- 🛠️ Finetune Microsoft's Phi-2, code
- 🛠️ Finetune Mamba, code
- 🛠️ Finetune Llama2 and Mistral using QLoRA, code
- ⚖️ Evaluate LLM language capabilities with meta's Belebele benchmark, code
- ⚖️ Evaluate LLM language capabilities with BLEU, code
- ⚖️ Llama2-70B as a judge of LLMs performs almost as good as GPT-4, code
- ⚖️ Validation loss is not a good metric for chatbot quality
- ⚖️ Use GPT3.5 as a judge of open-source LLMs, code
- 🛠️ Finetune Llama on podcast transripts with QLoRA, code
- 💅 Use Stable Diffusion for sketch-guided image generation, code