Technology ❯ Computer Science ❯ Artificial Intelligence

Model Training

Performance Evaluation Reinforcement Fine-Tuning Pre-training Low-Rank Adapters Pre-training Techniques Evaluation Metrics

4 ARTICLES

7mo ago

Studies Unveil Scaling Laws and Edge Quantization for Mixture-of-Experts Language Models

Researchers validated a metric for predicting sparse model compute efficiency, developing Hessian-aware low-bit inference with expert offloading to reduce on-device memory by roughly 60%