Xnhyacinth/Awesome-LLM-Long-Context-Modeling

★ 2,171⑂ 0

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

About Xnhyacinth/Awesome-LLM-Long-Context-Modeling

Xnhyacinth/Awesome-LLM-Long-Context-Modeling is an open-source project on GitHub, mainly written in several languages. 📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥 It currently holds 2,171 stars and 0 forks with 0 open issues, and was last pushed on an unknown date (repository created unknown).

Project Overview

AI Homed tracks it on the Today's Trending board.

GitHub Repository Details

Repository Xnhyacinth/Awesome-LLM-Long-Context-Modeling · default branch - · size 0 KB · watchers 0 · source: GitHub REST API and repository README

README

Large Language Model Based Long Context Modeling Papers and Blogs

📝 Survey Paper | 📄 Paper List | 🏠 Homepage | 📚 Notes | ⭐ GitHub

This repository curates papers and blogs on long-context language modeling, covering surveys; efficient attention; KV-cache optimization; recurrent transformers and state-space models; position encoding & length extrapolation; long-context training; long-term memory; retrieval-augmented generation; in-context learning; context and model compression; long reasoning (long CoT); long video & image; long-horizon agents; long-text generation; inference acceleration; benchmarks & evaluation; and technical reports.

🔥 Must-read papers for LLM-based Long Context Modeling.

🔥⚡🔥 Thanks for all the great contributors on GitHub!

🚀🤝🚀 I have the privilege of joining [LCLM-Horizon] and collaborating with them on providing a very complete and comprehensive scholarly survey \(A Comprehensive Survey on Long Context Language Modeling\) and repository \(A-Comprehensive-Survey-For-Long-Context-Language-Modeling\) dedicated to Long Context Language Modeling. I look forward to collaborating with them to advance research and deepen understanding in this area!

Taxonomy at a glance

flowchart LR
  LCLM["Long-Context Modeling"]
  LCLM --> A["Attention & KV Cache"]
  LCLM --> T["Training & Alignment"]
  LCLM --> M["Memory & RAG"]
  LCLM --> C["Compression"]
  LCLM --> R["Reasoning & Generation"]
  LCLM --> V["Multimodal / Video"]
  LCLM --> E["Evaluation & Acceleration"]
A --> A1["Sparse / Linear / IO-aware Attention"]
  A --> A2["Eviction / Quantization / Offloading"]
  T --> T1["Continual Pretraining / Long-SFT"]
  T --> T2["Adaptation & RL for Long Context"]
  M --> M1["Long-Term Memory"]
  M --> M2["RAG / Hybrid Long-Context"]
  C --> C1["Context Compression"]
  C --> C2["Model Compression"]
  R --> R1["Long CoT"]
  R --> R2["Long-Form Text Generation"]

If you find our repository and survey useful for your research, please consider citing the following paper:

@article{liu2025comprehensive,
  title={A Comprehensive Survey on Long Context Language Modeling},
  author={Liu, Jiaheng and Zhu, Dawei and Bai, Zhiqi and He, Yancheng and Liao, Huanxuan and Que, Haoran and Wang, Zekun and Zhang, Chenchen and Zhang, Ge and Zhang, Jiebin and others},
  journal={arXiv preprint arXiv:2503.17407},
  year={2025}
}