@AhmedFalih-kj3tt
we want more like this content!!! keep going man! 😊😊💖💖
@thedevlebowski
This is something I have been interested in for quite awhile but haven't found any great resources to really sink my teeth into. Thanks for the great and informative content.
@flwi
Great overview video! Thanks for putting in the time to create such a nice gem!
@johnshelbyjenkins
I was so hyped when you showed my llm_client crate! Kalosm is great though!
@letsgetrusty
Peak Rust content 🔥🦀
@StephenBlum
Nice! Great to learn more Rust libs with LLM support. Kalosm 🎉
@virusblitz
Wow, there really are so many applications! Amazing video❤
@amr3162
This video feels shinier than usual.
@AhmedFalih-kj3tt
Very good crate!! nice video
@northicewind
Awesome content! Excited to try this crate. Thank you for sharing
@marcof1430
Python is the standard for Data science and ML (but technically all the big frameworks Torch, Keira's+TF are in other languages), but in data preparation, in production, Rust can say something. For in memory data Polars is really great, also I used in a project tokenization in Rust and it's impressive.
@ian.mubangizi
This is brilliant 🎉😮
@DavidChoiProgrammer
Such a good video!
@first-thoughtgiver-of-will2456
if you are an engineer and serious about LLMs in Rust learn Candle. Ive been able to research and develop optimizers and architectures with Candle. Its an involved framework but is feature rich and complete in more places than any other framework. It makes design decisions that are reminiscent of torch and tensorflow and is very low level and not polyglot compared to alternatives.
@DaM_Cdn
I think the sweet spot for using a library like Kalosm would be integrating it into a Tauri app and downloading + executing the ML model on the user's local machine. Pair Kalosm & Tauri with a Leptos frontend, and you have a really compelling native desktop (or mobile) app! An installable PWA using a similar stack (minus Tauri of course) would also be a great use case for Kalosm + Leptos..
@__sassan__
Technically if you want to deploy an LLM powered application you would not just rely on a library like Transformers. You also need a performant server that can A. Do adaptive batching and B. Implement efficient “multiplications” to work optimally on multiple batches. So while kalosm looks dope, you would not want to use Rust if you actually need a python library like vLLM or NVIDIA’s triton (it’s oss). Folks at Kyutai (the company founded by the same guy who contributes to Candle) said they use rust for inference. So they must have implemented all of that in rust :) But if you don’t need that much load, of course use kalosm. I’m pretty sure you can find a crate that implements batching for tokio.
@oof-software
Here's an interesting paper on imposing format restrictions on LLMs: "Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models"
@liamwoodleigh
Great video! A shame there’s no structure output for the LLM-as-Service companies like OpenAI. Thanks for distilling all this!
@mahor1221
Thank you for the great video!
@dreamsofcode