Learning to Plan Long-Term for Language Modeling
Published in arXiv, 2024
Recommended citation: Florian Mai, Nathan Cornille, Marie-Francine Moens. (2024). "Learning to Plan Long-Term for Language Modeling." arXiv:2409.00070. https://arxiv.org/abs/2409.00070
We propose a planner that predicts a latent plan for many sentences into the future, allowing language models to trade computation time for better next token prediction accuracy.