👋 Welcome to Tile Language

GitHub

Tile Language (tile-lang) is a concise domain-specific language designed to streamline the development of high-performance GPU/CPU kernels (e.g., GEMM, Dequant GEMM, FlashAttention, LinearAttention). By employing a Pythonic syntax with an underlying compiler infrastructure on top of TVM, tile-lang allows developers to focus on productivity without sacrificing the low-level optimizations necessary for state-of-the-art performance.

GET STARTED

• Installation Guide
  • Installing with pip
  • Building from Source
  • Install with Nightly Version
• The Tile Language: A Brief Introduction
  • Programming Interface
  • Programming Interfaces
  • Compilation Flow
  • Tile-based Programming Model

TUTORIALS

• Writing High-Performance Kernels with the Tile Library
• Writing High-Performance Kernels with Thread Primitives
• Annotate Memory Layout
• Debugging Tile Language Programs
• Auto-Tuning Techniques for Performance Optimization
• Just In Time Compilation
• Pipelining Computation and Data Movement

DEEP LEARNING OPERATORS

• ElementWise Operators
• General Matrix-Vector Multiplication (GEMV)
• Triton implementation
• Naive Implementation in TileLang
• More concurrency
• Vectorized Reads
• tvm_thread_allreduce Instead of atomicAdd
• Autotune
• Conclusion
• General Matrix-Matrix Multiplication with Tile Library
• General Matrix-Matrix Multiplication with Dequantization
• Flash Attention
• Flash Linear Attention
• Convolution
• TMAC: Look Up Table Based Mixed Precision Computing

LANGUAGE REFERENCE

• Tile Language AST
• Tile Language: Primitives
• Tile Language: TileLibrary
  • T.Kernel
  • T.alloc_shared
  • T.alloc_fragment
  • T.copy
  • T.gemm
  • T.reduce_max T.reduce_sum
  • T.Parallel
  • T.Pipelined
  • T.clear T.fill
  • T.use_swizzle

API Reference

• Python API

Privacy

• Privacy