we want more like this content!!! keep going man! 😊😊💖💖
Peak Rust content 🔥🦀
This is something I have been interested in for quite awhile but haven't found any great resources to really sink my teeth into. Thanks for the great and informative content.
Great overview video! Thanks for putting in the time to create such a nice gem!
Nice! Great to learn more Rust libs with LLM support. Kalosm 🎉
Wow, there really are so many applications! Amazing video❤
Python is the standard for Data science and ML (but technically all the big frameworks Torch, Keira's+TF are in other languages), but in data preparation, in production, Rust can say something. For in memory data Polars is really great, also I used in a project tokenization in Rust and it's impressive.
I was so hyped when you showed my llm_client crate! Kalosm is great though!
Very good crate!! nice video
This video feels shinier than usual.
Awesome content! Excited to try this crate. Thank you for sharing
This is brilliant 🎉😮
Such a good video!
if you are an engineer and serious about LLMs in Rust learn Candle. Ive been able to research and develop optimizers and architectures with Candle. Its an involved framework but is feature rich and complete in more places than any other framework. It makes design decisions that are reminiscent of torch and tensorflow and is very low level and not polyglot compared to alternatives.
Excellent 👍🏽
Thank you for the great video!
Technically if you want to deploy an LLM powered application you would not just rely on a library like Transformers. You also need a performant server that can A. Do adaptive batching and B. Implement efficient “multiplications” to work optimally on multiple batches. So while kalosm looks dope, you would not want to use Rust if you actually need a python library like vLLM or NVIDIA’s triton (it’s oss). Folks at Kyutai (the company founded by the same guy who contributes to Candle) said they use rust for inference. So they must have implemented all of that in rust :) But if you don’t need that much load, of course use kalosm. I’m pretty sure you can find a crate that implements batching for tokio.
I think the sweet spot for using a library like Kalosm would be integrating it into a Tauri app and downloading + executing the ML model on the user's local machine. Pair Kalosm & Tauri with a Leptos frontend, and you have a really compelling native desktop (or mobile) app! An installable PWA using a similar stack (minus Tauri of course) would also be a great use case for Kalosm + Leptos..
Here's an interesting paper on imposing format restrictions on LLMs: "Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models"
@dreamsofcode