Two new papers propose tree-based speculative decoding

DominoTree and Trees from Marginals introduce conditional tree-structured drafting and factorized prior autoregressive drafting for faster LLM inference. Both aim to improve upon block-diffusion drafters by modeling joint token distributions.