Semiconductor Engineering

Dynamic KV Cache Scheduling in Heterogeneous Memory Systems for LLM Inference (Rensselaer Polytechnic Institute, IBM)

A new technical paper titled “Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System” was published by researchers at Rensselaer Polytechnic Institute and IBM. “Large ...
VentureBeat

How attention offloading reduces the costs of LLM inference at scale

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Rearranging the computations and hardware used to serve large language ...
Semiconductor Engineering

Pooling CPU Memory for LLM Inference With Lower Latency and Higher Throughput (UC Berkeley)

“The rapid growth of LLMs has revolutionized natural language processing and AI analysis, but their increasing size and memory demands present significant challenges. A common solution is to spill ...
manilatimes

PEAK:AIO Solves Long-Running AI Memory Bottleneck for LLM Inference and Model Innovation with Unified Token Memory Feature

Manchester, UK, May 19, 2025 (GLOBE NEWSWIRE) -- PEAK:AIO, the data infrastructure pioneer redefining AI-first data acceleration, today unveiled the first dedicated solution to unify KVCache ...
The Next Platform

Skimpy HBM Memory Opens Up The Way For AI Inference Memory Godbox

Generative AI is arguably the most complex application that humankind has ever created, and the math behind it is incredibly complex even if the results are simple enough to understand. GenAI also it ...
Yahoo Finance

Pliops Unlocks GenAI Inference with Smarter, Persistent Memory at FMS 2025

Pliops’ XDP LightningAI with FusIOnX stack transforms AI inference by unlocking real-time memory reuse. This enables smarter collaboration and delivers scalable performance with seamless deployment.
Yahoo Finance

Enfabrica Unveils Industry’s First Ethernet-Based AI Memory Fabric System for Efficient Superscaling of LLM Inference

Elastic Networked-Memory Solution Delivers Multi-800GB/s Read-Write Throughput Over Ethernet and Up To 50% Lower Cost Per Token Per User in AI Inference Workloads MOUNTAIN VIEW, Calif., July 29, 2025- ...
Forbes

Cerebras Now The Fastest LLM Inference Processor; Its Not Even Close

The company tackled inferencing the Llama-3.1 405B foundation model and just crushed it. And for the crowds at SC24 this week in Atlanta, the company also announced it is 700 times faster than ...
Design-Reuse

Silicon Photonic Chiplets with In-memory Computing Accelerate LLM Inference, Surpassing H100 Efficiency

Large language models demand ever-increasing computational resources, and current systems struggle with the communication bottlenecks that limit performance. Yue Jiet Chong, Yimin Wang, and Zhen Wu, ...
Business Wire

Enfabrica Unveils Industry’s First Ethernet-Based AI Memory Fabric System for Efficient Superscaling of LLM Inference

MOUNTAIN VIEW, Calif.--(BUSINESS WIRE)--Enfabrica Corporation, an industry leader in high-performance networking silicon for artificial intelligence (AI) and accelerated computing, today announced the ...