7.9 Seq2seq tasks

What this is and why it exists

Text in, text out — translation, summarisation, rewriting, question answering, and a great many things that were once separate research areas with separate architectures. This topic covers how that unification happened, the automatic measures the field uses and what they are blind to, and the failure that makes abstractive summarisation genuinely risky: a summary can be fluent, well-organised and confidently assert things the source never said.

The vocabulary

• Sequence-to-sequence — a task mapping one sequence to another of a different length.
• Reference — a human-produced correct output, used for automatic scoring.
• n-gram overlap — how many short word sequences the output and the reference share.
• Extractive — a summary built by selecting sentences from the source.
• Abstractive — a summary written in new words.
• Faithfulness — whether every statement in the output is supported by the source.
• Confabulation — a fluent, confident assertion with no basis in the input.
• Text-to-text framing — expressing every task as text in and text out.

The mental model

Translation is the task that built the field's habits, including its evaluation habits. Automatic scoring compares the output with one or more human reference translations by counting how many short word sequences they share, adjusted so that padding the output with common phrases does not help and so that outputs much shorter than the reference are penalised. It is fast, cheap, reproducible, and correlated with quality well enough to guide development.

Its weaknesses are documented and worth knowing precisely. A correct translation using different words than the reference scores badly, and there are usually many correct translations. The measure counts word sequences and cannot tell a fluent sentence from a scrambled one with the same words. It is insensitive to errors that matter enormously — a dropped negation changes the meaning entirely and barely moves the score. And the numbers are comparable only within one test set with one set of references and one tokenization, so a score quoted without those is not a number you can compare against anything. Use it as a development signal, not as evidence of quality, and report the settings alongside it.

Summarisation splits into two genuinely different tasks. Extractive summarisation selects sentences from the source and puts them together. It can be clumsy, repetitive and disjointed, and it has one enormous property: every sentence is verbatim from the source, so it cannot assert something the source did not say. Abstractive summarisation writes new text, which reads far better, compresses much more effectively, and can be wrong in ways the extractive approach structurally cannot.

That failure is the central risk of the topic and it deserves care. A model trained to produce fluent, plausible summaries will produce fluent, plausible summaries — including where the source is ambiguous, unusual, or contradicts what the model learned during training. So a name gets attached to the wrong person, an amount is rounded into a different amount, a hedge becomes a statement, a date is confidently wrong, or a conclusion the source explicitly declined to draw appears drawn. The output reads exactly like the correct output, which is what makes it dangerous: there is no clumsiness to warn the reader, and a reader who had time to check the source would not have needed the summary.

Overlap measures do not detect this at all. A confabulated summary shares most of its vocabulary with the source and scores well. Faithfulness needs its own check, and there are three practical approaches. Entailment checking: split the summary into claims and, for each, ask whether the source supports it, using a model trained for that judgement. Question-based checking: generate questions from the summary, answer them against the source, and compare with what the summary said. And human review on a sample, with reviewers asked specifically "is every statement here supported by the source", which is a different and much more answerable question than "is this a good summary".

Two design responses reduce the exposure. Require citation — have the system point at the source sentences supporting each part, which makes checking cheap and makes unsupported statements visible. And prefer extractive or lightly abstractive output wherever being wrong is expensive: in medical, legal or financial summarisation, the ugliness of verbatim sentences is a small price for the guarantee that nothing was invented.

The text-to-text framing is what unified all of this. Express every task as text in and text out, with the task itself described in the input — a prefix or an instruction — and one model with one objective handles translation, summarisation, classification and question answering alike. Classification becomes producing a class name as text; extraction becomes producing the extracted spans as text. The idea is simple and the consequence was large: instead of an architecture per task, one architecture, one training procedure, and a task specified in the input. That is directly the ancestor of instruction-following systems, where the task description is a natural-language request rather than a fixed prefix. It also inherits an awkwardness worth knowing — an output that must be parsed can be malformed, so any production pipeline needs validation and a fallback for output that does not match the expected shape.

The practical summary: build the extractive baseline first, because it is fast and cannot confabulate and gives you the number to beat. Report an overlap measure for development with its settings stated, and never as the headline. Measure faithfulness separately and explicitly. And choose the abstractive approach deliberately, knowing what it can do that the extractive one cannot — and what it can do that you do not want.

What you should now be able to explain or do

Describe how overlap-based scoring works and name four of its weaknesses. Say what must accompany such a score for it to be comparable. Distinguish extractive from abstractive summarisation by their failure modes, not only their outputs. Explain why confabulation is dangerous and why overlap measures cannot detect it. Apply the three faithfulness checks and the two design responses. State the text-to-text framing and what it unified, along with the parsing awkwardness it introduces.

Check yourself