RT @johnjnay: Gisting: 26x Compression of LLM Prompts

-Trains LLM to compress prompts into smaller sets of "gist" tokens to be reused for compute efficiency
-Can be easily trained as part of instruction fine-tuning
-FLOPs reductions, time speedups