2.4 Strings, formatting and regex

What this is and why it exists

Text cleaning is a daily chore in data work, and this topic gives you the two tools that do most of it: f-strings for producing readable output, and regular expressions for pulling structure out of messy input. It also settles the bytes-and-text distinction, which is the cause of the errors that appear only on somebody else's machine.

The vocabulary

• f-string — a string prefixed with f in which expressions inside braces are evaluated and inserted.
• Format spec — the part after a colon inside the braces, controlling width, decimals and alignment.
• str — text: a sequence of characters.
• bytes — raw data: a sequence of numbers from 0 to 255.
• Encoding — the rule mapping characters to bytes; UTF-8 is the one to use.
• Regular expression — a pattern describing a set of strings.
• Group — a parenthesised part of a pattern, captured separately.
• Quantifier — how many times the preceding thing may repeat.
• Anchor — a pattern element matching a position rather than a character.

The mental model

Start with the split that causes the most confusion. A str is text; a bytes object is data. They are not interchangeable, and the conversion between them always involves an encoding — a rule saying which bytes stand for which characters. Reading a file gives you bytes and something must decode them; if nobody says which encoding, Python uses a platform default, and that is why a script working on your machine fails on a colleague's with a decode error. The fix is to be explicit: pass encoding="utf-8" when you open a file, and treat any code that does not as a future bug report. Telugu, Hindi and Kannada text make this non-negotiable rather than tidy — a wrong encoding does not warn you, it produces the wrong characters.

f-strings are the output half and they do more than substitution. The format spec after the colon controls presentation: a fixed number of decimals, a width for alignment in a table, a thousands separator, a percentage. This is worth learning once because the alternative — rounding numbers by hand and padding with spaces — is where reporting code goes to die.

Regular expressions are the input half, and the model is: a pattern is a small program that walks the string. Literal characters match themselves; classes match a set; quantifiers repeat; groups capture; anchors pin the match to a position. The three functions you will use constantly are described precisely in the reference — search scans "looking for the first location where the regular expression pattern produces a match" and returns None if there is none; findall returns "all non-overlapping matches of pattern in string, as a list of strings or tuples"; and sub returns "the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement".

Then the trap, which the reference names: quantifiers are greedy. The star, plus and question mark "are all greedy; they match as much text as possible", and "adding ? after the quantifier makes it perform the match in non-greedy or minimal fashion; as few characters as possible will be matched". A greedy pattern intended to grab one field will happily swallow the rest of the line, and it does so silently — the match succeeds, so nothing warns you. Anchors are the other half of the defence: the caret "matches the start of the string" and the dollar matches "the end of the string" — and, where there is one, the position immediately before a trailing newline — so anchoring a pattern says that the whole thing must fit rather than some fragment of it.

The habit that follows is a rule for the rest of your career: test a pattern on hostile examples, not happy ones. Write down the input that is empty, the one with two of the thing, the one with the thing missing, the one with a comma inside the value. A regex that passes those is worth keeping; one tested only on the line you copied out of the file is a coin toss.

In code

Checked against the re reference and the string-formatting documentation.

import re

name, score, share = "asha", 88.4567, 0.732
print(f"{name:<10}{score:6.2f}  {share:.1%}")

line = "2026-08-27 WARN disk=91% host=node-3"

when = re.search(r"^(\d{4})-(\d{2})-(\d{2})", line)
if when:
    year, month, day = when.groups()

fields = dict(re.findall(r"(\w+)=([^\s]+)", line))
cleaned = re.sub(r"\s+", " ", line).strip()

Two things to read carefully. The date pattern is anchored with a caret, so it matches at the start of the line rather than anywhere a date-shaped thing appears. And the field pattern uses a class meaning "anything that is not whitespace" rather than a bare dot-star, because a greedy dot-star after the equals sign would take the rest of the line and leave you with one field instead of two.

What you should now be able to explain or do

Explain the difference between text and bytes, and name the one argument that prevents most encoding errors. Format a number to two decimals and a proportion as a percentage without arithmetic of your own. Say what search, findall and sub each return. Explain greedy matching and demonstrate the fix. Anchor a pattern and say what that changes. List four hostile inputs you would test any new pattern against.

Check yourself