LocalAI LLM Testing: i9 CPU vs Tesla M40 vs 4060Ti vs A4500

Sitting down to run some tests with i9 9820x, Tesla M40 (24GB), 4060Ti (16GB), and an A4500 (20GB)

Rough edit in lab session

Our website: robotf.ai/
Machine specs here: robotf.ai/Machine_Lab_Specs

GPUs being tested: (These are affiliate-based links that help the channel if you purchase from them!)
Telsa M40 amzn.to/3Yf4yXC
4060ti 16GB amzn.to/3NeSEGT
RTX A4500 20GB amzn.to/3TXtAYR

GPU Bench Node Components: (These are affiliate-based links that help the channel if you purchase from them!)
Open Air Case amzn.to/3U08Y27
30cm Gen 4 PCIe Extender amzn.to/3Unhclh
20cm Gen 4 PCIe Extender amzn.to/4eEiosA
1 TB NVME amzn.to/4gWFcFb
Corsair RM850x amzn.to/3NkITa4
128GB Lexar SSD amzn.to/3TZYYGh
G.SKILL Ripjaws V Series DDR 64GB Kit amzn.to/4dAZrWm
Core I9 9820x amzn.to/47UuIST
Nocuta NH-U12DX i4 CPU Cooler: amzn.to/3TZ7O6R
Supermicro CX299-PGF Logic Board amzn.to/3BxbWVr
Remote Power Switch amzn.to/3BubQOg

Recorded and best viewed in 4K
Your results may vary due to hardware, software, model used, context size, weather, wallet, and more!