PyTorch
Articles about PyTorch
Session-Aware Agentic Inference with NVIDIA Dynamo
TL;DR Agentic workloads change the traffic an inference server sees. Unlike single-turn chat, an agent session can involve a large initial prefill, repeated model calls, and subagents running in parallel,
Building Spyre as a Native PyTorch Device
TL;DR Spyre becomes a native PyTorch device by connecting PyTorch’s existing device, allocator, stream, and compiler abstractions through torch-spyre to the Spyre runtime and firmware. PrivateUse1 gives Spyre a real
Modernizing Table Batched Embeddings with FBTriton
This post explores the FBTriton kernel design for Table Batched Embedding (TBE) forward and backward passes. These core operators handle embedding lookups across thousands of sharded GPUs within recommendation systems.
Evolution of the PyTorch Media Processing Landscape
TL;DR If you need to decode or encode media, whether it’s images, video, or audio, use TorchCodec. If you need to transform media, use TorchVision for images and video, and
PyTorch Hardware Enablement: Updates from the Accelerator Integration Working Group
TL;DR The Accelerator Integration Working Group plays a vital role in standardizing how new hardware architectures connect to the open source AI ecosystem. As compute platforms diversify across cloud, edge,
Building a High-Performance and Portable vLLM Linear Backend with Helion
TL;DR We integrated Helion into vLLM’s linear backend to explore how an autotuned, high-level kernel DSL can improve LLM inference performance while reducing kernel implementation complexity. A single Helion general
New Pathway to PyTorch Certified Associate (PTCA) Certification
We’re excited to bring you the new PyTorch Certified Associate (PTCA) Certification Pathway, combining focused learning modules and the PyTorch Certified Associate (PTCA) certification exam together in one structured experience through Linux Foundation Education. The Certification Pathway is designed to help you build the foundational PyTorch skills validated by PTCA through learning that more closely aligns with the certification’s domains and competencies.
Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell
TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) — the attention kernel behind Meta’s Generative Ads Model (GEM) — on NVIDIA Blackwell (B200), built
A Ray-Focused Guide to PyTorch Conference North America
TL;DR In only a few weeks, PyTorch Conference North America 2026 will begin in San Jose, bringing together the open source AI community to share ideas and collaborate. If you’ve
From Upstream Changes to Downstream Confidence: Inside Torch Spyre’s Integration with PyTorch CRCR
TL;DR PyTorch’s Cross-Repository CI Relay (CRCR) gives out-of-tree accelerators a clean, scalable way to plug into upstream CI – and it leaves each backend free to decide which of PyTorch’s
Accelerate Your AI Journey with new Introduction Track at PyTorch Conference NA 2026 and PyTorch Associate Training
As deep learning models move rapidly from research prototypes into core enterprise infrastructure, the demand for practical, end-to-end PyTorch expertise has never been higher. Building robust neural networks requires more
From Research Project to Open Source Ecosystem: Bring Your Academic PyTorch Project to PyTorchCon NA
Some of the most interesting work being built with PyTorch starts in universities, research labs, student groups, and academic institutions. A new model architecture. A library created to support a
Hardware-Agnostic Models in vLLM
TL;DR To achieve state-of-the-art performance at the frontier, vLLM is changing its internal implementation in ways that make it incompatible with fullgraph torch.compile. This may have consequences for users who
How Shopify built a continual learning loop with PyTorch and vLLM
TL;DR: This case study explores how Shopify compresses production failures into model weights every day, beats frontier-model quality, and cuts serving costs 96% by building a continual learning loop with
TinyTorch: Don’t Just Import PyTorch. Build It
A framework you write yourself, tensors through transformers TL;DR Every mature systems project eventually needs a teaching version. TinyTorch is a free, open-source curriculum where you build a working ML
PyTorch Day Japan 2026 Comes to Tokyo on December 10
PyTorch Day Japan 2026 will bring the open source AI community together in Tokyo on December 10 for a full day of technical talks and interactive discussions designed to foster knowledge exchange and collaboration.
Open Research, Tooling & Optimization at PyTorch Conference North America 2026
TL;DR Taking place October 20 to 21 in San Jose, California, PyTorch Conference North America 2026 highlights open research, tooling, and performance optimization across compiler architecture, cross-hardware kernel domain-specific languages,
Low Precision Flash Attention 4: End-to-End Block-Scaled Attention for Blackwell
TL;DR We extend FlashAttention-4 [1] with MXFP8 forward and backward, reaching 2.85 PF/s forward and 2 PF/s backward on LLM shapes. On our internal shapes, FA4 MX8 reaches 2.54 PF/s