Articles by Tag #llminference
Browse our collection of articles on various topics related to IT technologies. Dive in and explore something new!
TorchAO vs ONNX Runtime: 8-bit Quantization Benchmark
TorchAO Just Beat ONNX Runtime on My M1 MacBook (And I Didn't Expect It) I ran the same...
HeMA-MISO: Heterogeneous Memory Architecture for LLM Inference with SW Optimization
Note: This research was conducted in the first half of 2025. Some information may be outdated at the...
Fine-Tuning and Inference in LLMs: From Custom Models to Production Deployment
As organizations move beyond using pre-trained models, two critical concepts become essential:...
Paper Review: GQA — Grouped Query Attention for Faster LLM Inference
Why Multi-Head Attention Has a Memory Problem Here's a number that might surprise you: in...
Speculative Decoding: Why 2x Faster Inference Fails
The Promise That Breaks Under Load Speculative decoding claims to make LLM inference 2-3x...