If you're getting errors deploying the model on the GPU, set the --enforce-eager in the docker commands. Good luck!
@jmanhype1
7 ай бұрын
amazing yet again. leading innovation. trendsetting!
@timothylenaerts1123
6 ай бұрын
can do a call to v1/models and just dynamically pull the model name
@NicolasEmbleton
8 ай бұрын
Nicely put together. I've used vLLM with serverless, but it's quite a bit harder with all the parameters such as concurrency and GPUs and such. I'll give a try to this method see what gives.
@Data-Centric
8 ай бұрын
Thanks, I might do one on serverless
@CemizBont
8 ай бұрын
Ver nice and comprehensive tutorial. Will give it a try. thank you jhon! Btw. i love the Alis picture behind you 😍
@Data-Centric
8 ай бұрын
Thanks, and you’re welcome; let us know how it goes!
Пікірлер: 8