Is TinyLLaMA any good? Let's run it on Raspberry Pi, benchmark the baseline and possible optimizations! Then we talk with our TinyLLaMA living inside of Raspberry Pi using a simple barebones web server for inference.
Github Gist (benchmark commands, benchmark results, prompts):
gist.github.co...
TinyLlama/TinyLlama-1.1B-Chat-v1.0
huggingface.co...
TinyLlama/TinyLlama-1.1B-Chat-v1.0 (GGUF)
huggingface.co...
Негізгі бет Run this Small Language Model on Raspberry Pi for Fun and PROFIT
Пікірлер: 19