AirLLM runs 70B models on a single 4GB GPU

AirLLM performs 70B-parameter inference on a single 4GB GPU by streaming one layer at a time instead of loading the whole model into memory. The technique is pitched as the next step after quantization, letting users skip renting an A100.