Skip to main content

9. The Abstract

The abstract is read by far more people than the rest of the paper — most of the field will read it and nothing else. It is also the first thing reviewers see, and many form a preliminary judgment from it before reading anything else. Spend disproportionate effort here.

The five-move abstract​

A reliable structure for an *ACL abstract, in roughly this order, in roughly 150–250 words:

  1. Context (1–2 sentences). What problem space are we in? What is the gap or open question?
  2. Move (1 sentence). "In this paper, we…" — the specific thing you did.
  3. Method (1–3 sentences). How you did it, at a level a non-specialist can follow. No equations. No undefined acronyms.
  4. Result (1–3 sentences). The headline finding, with the most informative single number if applicable. Be specific: "improves accuracy by 4.2 points on MMLU" beats "improves performance significantly."
  5. Implication (1 sentence). Why the reader should care. What does the field now know that it did not?

Optional 6th sentence — Release statement. "Our code and models are available at https://…" or "We release the dataset, splits, and baselines at …" Increasingly expected by reviewers, especially for resource, benchmark, and toolkit papers, and scored under most *ACL venues' reproducibility criterion. Keep it short, put it last. It is a separate move from Implication: Implication tells the reader why the work matters; the release statement tells them where to get the artifact. For the anonymous-submission version, "Code and data will be released upon acceptance" is the standard placeholder. The phrasebook below has a Release / reproducibility statements subsection with real examples.

Phrasebook: phrases for each move​

Each subsection below lists verbatim openings, pivots, and closings from highly-cited papers — exactly the parts of an abstract where new authors tend to produce filler.

A note on the count. The five-move structure above maps to seven phrasebook subsections, not five. We split Context into Context openers and Gap statements — a single sentence often does both jobs (RoBERTa's opener, "Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging," is context and gap fused inside one sentence) but the two jobs are rhetorically distinct, and many abstracts spread them across two or three sentences. We also add a seventh subsection, Release / reproducibility statements, covering the optional 6th sentence flagged in the previous section. Study them as separate patterns; then decide per-paper whether to fuse or split, and whether the release sentence belongs in your abstract or only in the body.

These are patterns to internalise, not templates to fill in word-for-word. Lifting a phrase verbatim is plagiarism; absorbing the rhetorical move — "the 'however + enumerated shortcomings' pivot," "the move sentence that does double duty as a method summary" — and writing your own version is craft. Read a subsection, notice the underlying move, close the page, then write. The glosses in italics describe the move so you can name it; the quotes are evidence that the move works in real published papers.

The same move recurs at three scales across this guide: the title compresses it to a phrase, the abstract states it in a sentence, and the introduction unfolds it into a paragraph. Reading the three phrasebooks side by side — often drawn from the very same papers — is the fastest way to see one piece of rhetoric expand and contract to fit its slot.

Under each move below, a couple of examples sit in the open so you can skim the pattern at a glance; expand Show more for the rest.

Context openers​

  • "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration." — Opens by naming the incumbent paradigm in one sentence; sets up the dethroning. Attention Is All You Need (Vaswani et al., NeurIPS 2017)
  • "Neural sequence-to-sequence models have provided a viable new approach for abstractive text summarization (meaning they are not restricted to simply selecting and rearranging passages from the original text)." — Opener that names the approach and parenthetically defines the key term. Get To The Point: Summarization with Pointer-Generator Networks (See et al., ACL 2017)
Show 10 more

Gap statements​

Show 11 more
  • "However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures." — Gap framed as a capability ceiling on a specific task class. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., NeurIPS 2020)
  • "However, at some point further model increases become harder due to GPU/TPU memory limitations and longer training times." — Gap as a hard resource constraint, not a quality complaint. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (Lan et al., ICLR 2020)
  • "However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT." — Gap quantified with a concrete cost number. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (Reimers & Gurevych, EMNLP 2019)
  • "By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do." — "By contrast" pivot via a human-vs-model gap. Language Models are Few-Shot Learners (Brown et al., NeurIPS 2020)
  • "While promising, this approach requires to use models with billions of parameters, which are expensive to train and query." — "While promising" concession-pivot; quantifies the cost. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (Izacard & Grave, EACL 2021)
  • "Nevertheless, they depend heavily on human-written instruction data that is often limited in quantity, diversity, and creativity, therefore hindering the generality of the tuned model." — Three-fold gap (quantity/diversity/creativity) tied to a downstream consequence. Self-Instruct: Aligning Language Models with Self-Generated Instructions (Wang et al., ACL 2023)
  • "They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel." — Gap framed as a paradox to grab attention. Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al., NeurIPS 2023)
  • "However, RLHF is a complex and often unstable procedure, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model." — Gap expanded into a mini-explanation of why the prior method is painful. Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., NeurIPS 2023)
  • "In contrast, most NLU models above the word level are designed for a specific task and struggle with out-of-domain data." — "In contrast" pivot from the opening claim about humans. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (Wang et al., EMNLP Workshop 2018)
  • "However, machine learning research in this area has been dramatically limited by the lack of large-scale resources." — Gap as a resource bottleneck, motivating a dataset paper. A large annotated corpus for learning natural language inference (Bowman et al., EMNLP 2015)
  • "Despite its importance to experimental design, statistical power (the probability that, given a real effect, an experiment will reject the null hypothesis) has largely been ignored by the NLP community." — "Despite" pivot pointing at a community-wide oversight. With Little Power Comes Great Responsibility (Card et al., EMNLP 2020)

Move / proposal statements​

Show 12 more
  • "We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely." — "Propose + name + key design choice + what it removes" in one sentence. Attention Is All You Need (Vaswani et al., NeurIPS 2017)
  • "We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy)." — Enumerated contribution claim baked into the move sentence. Deep Contextualized Word Representations (Peters et al., NAACL 2018)
  • "We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks." — Move sentence does double duty as a method summary. LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., ICLR 2022)
  • "We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations." — Names, categorises ("framework"), and gives the mechanism. Self-Instruct: Aligning Language Models with Self-Generated Instructions (Wang et al., ACL 2023)
  • "In this paper, we introduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages." — Move sentence carries the scaling number (101 languages). mT5: A massively multilingual pre-trained text-to-text transformer (Xue et al., NAACL 2021)
  • "We propose BLEURT, a learned evaluation metric for English based on BERT." — Eleven words: name, type, language, base. BLEURT: Learning Robust Metrics for Text Generation (Sellam et al., ACL 2020)
  • "We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction." — Enumerates the four capabilities of the new model in the move sentence. Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al., NeurIPS 2023)
  • "We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100,000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question is a segment of text from the corresponding reading passage." — Move sentence packed with all key facts (size, source, format). SQuAD: 100,000+ Questions for Machine Comprehension of Text (Rajpurkar et al., EMNLP 2016)
  • "To facilitate research in this direction, we present the General Language Understanding Evaluation (GLUE, gluebenchmark.com): a benchmark of nine diverse NLU tasks, an auxiliary dataset for probing models for understanding of specific linguistic phenomena, and an online platform for evaluating and comparing models." — Move sentence enumerates the three deliverables of the benchmark paper. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (Wang et al., EMNLP Workshop 2018)
  • "This paper introduces the Multi-Genre Natural Language Inference (MultiNLI) corpus, a dataset designed for use in the development and evaluation of machine learning models for sentence understanding." — Classic "this paper introduces X, a Y for Z" template. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (Williams et al., NAACL 2018)
  • "We propose two novel model architectures for computing continuous vector representations of words from very large data sets." — Move sentence frames the contribution as plural (two architectures). Efficient Estimation of Word Representations in Vector Space (Mikolov et al., ICLR 2013)
  • "Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches." — "Here we show" framing turns the contribution into a finding rather than an artefact. Language Models are Few-Shot Learners (Brown et al., NeurIPS 2020)

Method summaries​

Show 11 more
  • "Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector." — Encoder-decoder mechanism described in one sentence. Sequence to Sequence Learning with Neural Networks (Sutskever et al., NeurIPS 2014)
  • "First, we use a hybrid pointer-generator network that can copy words from the source text via pointing, which aids accurate reproduction of information, while retaining the ability to produce novel words through the generator." — "First/Second" structure to lay out two mechanisms. Get To The Point: Summarization with Pointer-Generator Networks (See et al., ACL 2017)
  • "In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units." — Mechanism summarised with the design rationale embedded. Neural Machine Translation of Rare Words with Subword Units (Sennrich et al., ACL 2016)
  • "Our approach combines a search component based on bigram hashing and TF-IDF matching with a multi-layer recurrent neural network model trained to detect answers in Wikipedia paragraphs." — Method summary names both modules and how they connect. Reading Wikipedia to Answer Open-Domain Questions (Chen et al., ACL 2017)
  • "Instead of masking the input, our approach corrupts it by replacing some tokens with plausible alternatives sampled from a small generator network." — "Instead of X, our approach does Y" — contrastive method summary. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators (Clark et al., ICLR 2020)
  • "In this paper we introduce a new parameterization of the reward model in RLHF that enables extraction of the corresponding optimal policy in closed form, allowing us to solve the standard RLHF problem with only a simple classification loss." — Mechanism stated as a reformulation of a known problem. Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., NeurIPS 2023)
  • "In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information." — Method summary with the design rationale spelled out for both halves. ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., ICLR 2023)
  • "Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model." — One-sentence pipeline summary: generate → filter → finetune. Self-Instruct: Aligning Language Models with Self-Generated Instructions (Wang et al., ACL 2023)
  • "We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever." — Names the two memories and their concrete instantiations. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., NeurIPS 2020)
  • "This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time." — Method summary via taxonomy of two mechanisms. Effective Approaches to Attention-based Neural Machine Translation (Luong et al., EMNLP 2015)
  • "In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity." — Names the modification, the structure, and the resulting operation. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (Reimers & Gurevych, EMNLP 2019)

Result statements​

  • "It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5 (7.7 point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement)." — Headline number plus four datasets, with absolute deltas in parentheses. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., NAACL 2019)
  • "Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU." — Number + dataset + delta in one sentence. Attention Is All You Need (Vaswani et al., NeurIPS 2017)
Show 11 more
  • "We apply our model to the CNN / Daily Mail summarization task, outperforming the current abstractive state-of-the-art by at least 2 ROUGE points." — Result framed as a minimum delta ("at least 2"). Get To The Point: Summarization with Pointer-Generator Networks (See et al., ACL 2017)
  • "When evaluated on a wide range of open-domain QA datasets, our dense retriever outperforms a strong Lucene-BM25 system greatly by 9%-19% absolute in terms of top-20 passage retrieval accuracy, and helps our end-to-end QA system establish new state-of-the-art on multiple open-domain QA benchmarks." — Headline result as an absolute range over a strong baseline. Dense Passage Retrieval for Open-Domain Question Answering (Karpukhin et al., EMNLP 2020)
  • "With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems which already incorporate known techniques such as dropout." — Result hedged-up by specifying the baseline already used good tricks. Effective Approaches to Attention-based Neural Machine Translation (Luong et al., EMNLP 2015)
  • "Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times." — Two efficiency numbers against a named baseline. LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., ICLR 2022)
  • "In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters." — Punchy comparison: smaller model preferred over 100x larger one. Training language models to follow instructions with human feedback (Ouyang et al., NeurIPS 2022)
  • "As a highlight, Chinchilla reaches a state-of-the-art average accuracy of 67.5% on the MMLU benchmark, greater than a 7% improvement over Gopher." — "As a highlight" frames the headline number explicitly. Training Compute-Optimal Large Language Models (Hoffmann et al., NeurIPS 2022)
  • "For instance, prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier." — Result with sample size (eight) baked into the sentence. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Wei et al., NeurIPS 2022)
  • "Applying our method to the vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT-001, which was trained with private user data and human annotations." — Numeric delta + comparison to a stronger (paid-data) baseline. Self-Instruct: Aligning Language Models with Self-Generated Instructions (Wang et al., ACL 2023)
  • "Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generates human-like task-solving trajectories that are more interpretable than baselines without reasoning traces." — "Concretely" pivot to grounded results across two named tasks. ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., ICLR 2023)
  • "We build a strong logistic regression model, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%)." — Number, baseline, delta — minimal but complete. SQuAD: 100,000+ Questions for Machine Comprehension of Text (Rajpurkar et al., EMNLP 2016)
  • "For machine translation, we find that typical test sets of 2000 sentences have approximately 75% power to detect differences of 1 BLEU point." — Result as a quantified diagnostic claim about the field's own evaluations. With Little Power Comes Great Responsibility (Card et al., EMNLP 2020)

Implication closers​

  • "We discuss broader societal impacts of this finding and of GPT-3 in general." — Closer signposts the impact discussion rather than overclaiming. Language Models are Few-Shot Learners (Brown et al., NeurIPS 2020)
  • "Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent." — Hedged closer: acknowledges limitation, then states what the field now knows. Training language models to follow instructions with human feedback (Ouyang et al., NeurIPS 2022)
Show 12 more
  • "These results highlight the importance of previously overlooked design choices, and raise questions about the source of recently reported improvements." — Closer that reframes prior literature, not just the present paper. RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019)
  • "Self-Instruct provides an almost annotation-free method for aligning pre-trained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning." — Closer pairs the claim with a concrete release. Self-Instruct: Aligning Language Models with Self-Generated Instructions (Wang et al., ACL 2023)
  • "However, human performance (86.8%) is much higher, indicating that the dataset presents a good challenge problem for future research." — Implication closer that positions the dataset as a future benchmark. SQuAD: 100,000+ Questions for Machine Comprehension of Text (Rajpurkar et al., EMNLP 2016)
  • "To improve the situation going forward, we give an overview of best practices for power analysis in NLP and release a series of notebooks to assist with future power analyses." — Closer that prescribes practice for the community. With Little Power Comes Great Responsibility (Card et al., EMNLP 2020)
  • "More importantly, we hope our approach opens up a productive line of inquiry in which different LLM architectures are tested on a variety of impossible languages in an effort to learn more about how LLMs can be used as tools for these cognitive and typological investigations." — Closer frames the contribution as opening a research programme. Mission: Impossible Language Models (Kallini et al., ACL 2024)
  • "For language generation tasks, we find that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline." — Closer that states the qualitative property (factuality), not just a number. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., NeurIPS 2020)
  • "This is evidence that sequence-to-sequence models offers a flexible framework to efficiently aggregate and combine evidence from multiple passages." — Closer frames the empirical win as evidence for a broader claim. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (Izacard & Grave, EACL 2021)
  • "Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities." — Closer balances the gain with what was preserved. Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al., NeurIPS 2023)
  • "We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data." — Implication closer demonstrates generality beyond the headline task. Attention Is All You Need (Vaswani et al., NeurIPS 2017)
  • "We also present an analysis showing that exposing the deep internals of the pre-trained network is crucial, allowing downstream models to mix different types of semi-supervision signals." — Closer surfaces an analysis finding, not just headline scores. Deep Contextualized Word Representations (Peters et al., NAACL 2018)
  • "Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition." — Closer adds qualitative evidence that the mechanism behaves as designed. Neural Machine Translation by Jointly Learning to Align and Translate (Bahdanau et al., ICLR 2015)
  • "Finally, we found that reversing the order of the words in all source sentences (but not target sentences) improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier." — Closer surfaces a counter-intuitive engineering finding for the field. Sequence to Sequence Learning with Neural Networks (Sutskever et al., NeurIPS 2014)

Release / reproducibility statements​

Anonymity warning. For double-blind *ACL submission, strip any release sentence that identifies you: GitHub URLs, lab pages, personal websites, named pip packages, even "anonymised" links whose slug still reveals authorship. Use a neutral placeholder — "Code and data will be released upon acceptance" is the standard form. Swap the full release sentence back in for the camera-ready version, after the paper is accepted. Most of the examples below are quoted from the camera-ready form; the URLs were not in the original anonymous submission.

Show 12 more

Common mistakes​

  • Generic openings. "Natural language processing has seen tremendous progress in recent years." Reviewers have read this sentence ten thousand times. Cut it. Start with the specific problem.
  • Burying the result. If your headline number is in the last sentence, move it up. Reviewers should know whether you won by sentence four.
  • Hedge-stuffing. "We hope to potentially demonstrate that our approach may show promise." No. "We show that…" If you cannot say show, your evidence is too thin.
  • Method-only abstracts. Telling me you used a transformer encoder is not the abstract; it is part of the methods section.
  • Mentioning what is in the paper without saying what you found. "We also conduct extensive ablations and analysis." Of course you did. Tell me what they revealed.

Drafting tip​

Write the abstract last. I know everyone says this and nobody does it. Do it. The abstract you write before the experiments are finished is a wish list; the abstract you write at the end is a summary. Reviewers can tell the difference.

Five examples from real ACL papers​

  1. BERT (Devlin et al., NAACL 2019). A model abstract: context (pretrained representations), move (bidirectional pretraining), method (masked LM + NSP), result (11 SOTA results, specific numbers), implication (a single fine-tuning recipe works broadly).
  2. Attention Is All You Need (Vaswani et al., 2017). Two sentences of context on RNN/CNN sequence models, one sentence move (Transformer), one method sentence, three sentences of results with specific BLEU scores.
  3. With Little Power Comes Great Responsibility (Card et al., EMNLP 2020). Opens with the gap, not the field. The first sentence does the work of three.
  4. GloVe (Pennington et al., EMNLP 2014). Notice how it positions itself against two prior families of methods, then states what its model captures that they miss.
  5. Mission: Impossible Language Models (Kallini et al., ACL 2024 Best Paper). The abstract reads like the puzzle it solves: setup, hypothesis, experiment, finding.

Exercise​

Read the abstracts of 20 best papers from the last three years of ACL. For each, label the five moves. You will quickly see which abstracts are great and which were rushed — and your own abstract will get sharper by osmosis.

Further reading​