P-3.3 Arrays, Strings and Buffer Overruns

What this is and why it exists

In C, an array is an address and a promise. A string is an array that ends with a zero byte.

Nothing records how long either one is. Every loop and every copy is trusting a count that lives somewhere else, in your head or in another variable.

That design is the direct ancestor of a large share of the security defects of the last forty years. It is worth understanding once at the level of the memory, rather than carrying around a vague instruction to be careful.

The vocabulary

• Array — elements stored one after another in memory.
• Decay — an array name used as a pointer to its first element.
• Index — arithmetic on that address, not a checked lookup.
• Null terminator — the zero byte marking the end of a string.
• Buffer — a block of storage a copy is written into.
• Overrun — writing past the end of that block.
• Bounded copy — a copy that takes a maximum length and respects it.

The mental model

Array elements sit one after another in memory, and the array's name is an address — it decays to a pointer to the first element. So indexing is arithmetic on that address. Element five is the start plus five element-widths.

That sentence explains something important. An index past the end is not an error. It is arithmetic that produces a valid-looking address which happens to be outside your array. Nothing checks it, because nothing knows where the array ends.

Strings inherit all of that and add one convention. The end is marked by a zero byte, and length is found by walking forward until that byte appears. Every string function in the standard library depends on it being there. A string missing its terminator has no end, and a function looking for one will keep walking through whatever comes next.

So what happens when a write goes past the last element? It lands on whatever was next in memory. That might be another variable. It might be bookkeeping the program relies on. It might be nothing important. Which of those it is can vary between runs and between machines, which is why these defects are so inconsistent.

And this is where it stops being only a correctness problem. If the data being copied comes from outside the program, then whoever supplies that data influences what gets written over. That is the mechanism behind a whole family of well-documented vulnerabilities. It is why bounds are treated as a security matter and not a tidiness one.

The defence is unglamorous and completely effective. Carry the length yourself, next to the storage it describes. Use the library functions that take a maximum. Check the destination is big enough before copying anything into it. None of that is clever, and cleverness is not what is wanted here.

What you should now be able to explain or do

Say what an array name actually is, and why indexing is arithmetic. Explain why an index past the end is not reported as an error. Say how a string's length is found and what happens when the terminator is missing. Describe what a write past the end lands on, and why the effect varies between runs. Explain why bounds are a security matter once the data comes from outside. Write a copy that carries a length and checks the destination first.

Check yourself