Skip to main content

30. Best Papers Worth Reading

Read these papers in full, past the abstract. Watch how each one is structured, motivates its problem, and presents results. Watch how it handles the boring sections: related work, experimental setup, limitations. Then hold your own writing up against them.

A BibTeX file of every paper in this chapter is available at examples/papers-to-read.bib. Drop it straight into Overleaf.

We group the papers by what each does best, so you can read with a purpose.

For learning how to frame a problem​

  • Vaswani et al., Attention Is All You Need (NeurIPS 2017). Honorary inclusion. It is outside *ACL but the canonical example of a focused contribution. Watch how the abstract and introduction commit to one claim and defend it.
  • Devlin et al., BERT (NAACL 2019, NAACL Best Long Paper). The introduction is a textbook case of "here's the gap, here's the move, here's the result." Read it three times.
  • Pennington, Socher, Manning, GloVe (EMNLP 2014, ACL Test-of-Time 2024). Positions itself against two prior traditions and stakes out the middle ground precisely.
  • Bender et al., On the Dangers of Stochastic Parrots (FAccT 2021). Field-defining problem framing. Graduate seminars study its introduction.

For learning how to do analysis​

For learning how to write methods​

  • Vaswani et al., Attention Is All You Need. (Yes, again.) The Methods section is a model of clear exposition; study it line by line.
  • See, Liu, Manning, Get To The Point (ACL 2017). Pointer-generator architecture explained with discipline.
  • Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020). Cleanly decomposes a complex method.
  • Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (NeurIPS 2022). The method is simple, and the paper presents it simply.

Recent best-paper winners worth studying (*ACL family)​

These are recent, on-topic, and exemplary in different ways. Find them on the ACL Anthology.

  • Kallini, Papadimitriou, Futrell, Mahowald, Potts, Mission: Impossible Language Models (ACL 2024 Best Paper). Beautiful experimental design, clean analysis, sharp claims.
  • Lu, Liu, Mortensen, Semisupervised Neural Proto-Language Reconstruction (ACL 2024 Best Paper). A paper rooted in linguistic theory and rigorous in NLP method.
  • Hahn, Rofin, Why Are Sensitive Functions Hard for Transformers? (ACL 2024 Best Paper). A theory paper that stays grounded in empirical observations.
  • Guan et al., Deciphering Oracle Bone Language with Diffusion Models (ACL 2024 Best Paper). First-authored by an undergraduate. A new idea plus careful execution wins, whoever the author is.
  • Groeneveld et al., OLMo: Accelerating the Science of Language Models (ACL 2024 Best Theme Paper). A resource/openness paper done right.
  • Subbiah, Zhang, Chilton, McKeown, Reading Subtext: Evaluating LLMs on Short Story Summarization with Writers (TACL 2024 / ACL 2025 Best Paper). Shows how to mix human-expert evaluation with NLP rigor.
  • Yao et al., VisKoP: Visual Knowledge oriented Programming for Interactive KBQA (ACL 2023 Best Paper). A system/demo paper that explains a complex tool clearly.
  • From EACL 2024 Best Papers: LOCOST: State-Space Models for Long Document Abstractive Summarization (Le Bronnec et al.); Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs (Balloccu et al.). Read both for how they handle contested questions of method.
  • From EMNLP 2023: Morris, Kuleshov, Shmatikov, Rush, Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023 Outstanding). A short, surprising paper with disciplined experimental design.
  • From ACL 2023 Best Papers: the awards page lists three best papers and 39 outstanding papers. Browse and pick three that overlap with your area.

Test-of-Time papers (papers that aged well)​

The ACL Test-of-Time Award honors papers from 10 and 25 years prior whose impact has endured. Read them to see what a "good idea, well executed" looks like in retrospect:

  • Pennington, Socher, Manning, GloVe (EMNLP 2014, 10-year award 2024).
  • Lillian Lee, Measures of Distributional Similarity (ACL 1999, 25-year award 2024).

How to read these​

One pass is not enough. For two or three papers from this list:

  1. Read the abstract. Try to predict the structure of the paper. Read it again.
  2. Read the introduction. Mark, paragraph by paragraph, what role each paragraph plays (context, gap, move, preview, contributions).
  3. Skim the methods. Look at the figures first. Can you understand the method from the figures alone?
  4. Read the results. Identify the single most important table or figure.
  5. Read the limitations. Notice what the authors chose to surface and how.
  6. Now read your own draft with the same lens.

This exercise teaches more than ten "tips for writing papers" blog posts.

Want to contribute a paper to this list?​

We welcome suggestions; see CONTRIBUTING.md. The bar: it must exemplify a specific craft point in a section of this guide, and it must be linkable on the ACL Anthology (or a comparable peer-reviewed venue).