How UBTree Speeds Up LLMs with a Tree of Draft Tokens

What if a language model could check several possible continuations at once? UBTree combines a parallel unigram proposer, a parallel bigram selector and a verification tree to speed up speculative decoding. This short visual explainer shows why a single wrong draft token wastes a path, how candidate transitions form a tree, and why the target model remains in control. Benchmark context: the authors report a 6.94× average decoding speedup on Qwen3-8B with greedy decoding across seven benchmarks, batch size 1. This is an author-reported research result—not a guarantee for every model, workload or device. Our token diagrams are illustrations, not live benchmarks. The supplied Ling3 Flash demo uses a different setup. Sources: Project: https://www.inclusion-ai.org/ubtree/ Paper: https://arxiv.org/abs/2609.39972v2 Draft model and serving details: https://huggingface.co/inclusionAI/Ling-3.0-flash-UBTree Models: https://huggingface.co/collections/inclusionAI/ubtree Research and source figures: Chumeng Liang et al., Inclusion AI, UIUC and Purdue. Architecture figure: CC BY 4.0, credited to the authors. Original explanatory animation, edit and sound bed; AI-generated narration. The MIT licence applies to the released draft-model weights. Matching target models are required; the drafts are not standalone chat models. At preparation, SGLang support uses a separate branch; upstream integration is in progress. #UBTree #AI #LLM