13.7 Coding and ML theory interviews

What this is and why it exists

Screening rounds are their own skill and they are practised in the wrong proportions. People spend months on competitive programming problems that machine learning screens rarely reach, and almost no time on the things asked constantly: manipulating a dataframe under time pressure, writing a standard algorithm from nothing, and explaining a concept rather than defining it. Practising the right level saves months, and this topic is what that level actually is.

The vocabulary

• Screen — an early round filtering candidates before the deeper interviews.
• Data structures and algorithms — the programming portion.
• Live coding — writing code while somebody watches, under time pressure.
• From scratch — implementing with array operations and no library that does it for you.
• Theory question — a conceptual question with no code.
• Explaining rather than defining — saying what something is for, not reciting its definition.

The mental model

The programming level is usually less extreme than reputation suggests, and believing otherwise costs people months.

What is actually asked: comfortable use of dictionaries, sets and lists; string and array manipulation; sorting with a custom key; counting and grouping; two pointers and a sliding window; a hash map to turn a quadratic solution into a linear one; recursion where it is natural; and being able to state the cost of what you wrote. That is a moderate level, it can be reached in a few weeks of practice, and it is where the questions in this field concentrate.

What is rarely asked in a machine learning screen: advanced dynamic programming, sophisticated graph algorithms, intricate tree balancing, and the difficult end of competitive programming. Those appear in general software screens, and grinding hundreds of them is the most common misallocation of preparation time in this field.

Practise the moderate level until it is comfortable and then stop, and spend what you saved on the three sections below.

Live data manipulation is where many otherwise strong candidates lose ground, because it is a performance skill rather than a knowledge one. You know how to group and aggregate; doing it in four minutes while somebody watches, without the documentation, is a different thing.

The operations that recur: filtering on several conditions; grouping and aggregating, including several aggregates at once; joining two tables and reasoning about what happens to unmatched rows; pivoting between long and wide; ranking within a group, which is the one people fumble most; handling missing values deliberately; working with dates — extracting parts, resampling, computing differences; and a rolling calculation over time.

In the query language: joins of every kind, aggregation with a filter on the aggregate, window functions for ranking and running totals, common table expressions to keep a query readable, and dates. Window functions are the single highest-value thing to practise, because they come up constantly and are the part people have usually avoided.

How to practise is the important part: with a timer, out loud, from memory, on a dataset you did not prepare. Twenty minutes a day for two weeks changes this completely, and reading about it changes nothing.

Implementing from scratch tests understanding rather than recall, and three come up repeatedly.

k-means: initialise centres, assign each point to the nearest, recompute each centre as the mean of its members, repeat until stable. Write it with array operations, and be ready for the follow-ups — how do you choose the number of clusters, what does a poor initialisation do, why does it prefer round clusters of similar size.

Logistic regression: the linear score, the squashing function, the loss, the gradient, the update loop. This one tests whether you understood the classical module, and the follow-ups are about the loss and about what regularisation changes.

Attention: project to queries, keys and values; multiply queries by transposed keys; divide by the square root of the key dimension; add the mask; softmax over the key positions; multiply by values. Six lines, and the follow-up is always why the division is there — because dot products grow with dimension, large scores make the softmax nearly one-hot, and its gradient there is almost zero, so learning stops.

Write each of the three from memory once a week for a month and they become automatic. They are asked because they cannot be answered by having read about the technique, which is exactly why they are worth the practice.

Theory questions check whether you can explain rather than define, and that distinction is the whole assessment.

Bias and variance: not the definitions, but which one you have from a described symptom, and what you would do about each. A model excellent on training data and poor on held-out data is variance; both poor is bias; and the responses differ completely.

Regularisation: what a penalty on the weights prefers, why one form drives coefficients to exactly zero and another shrinks them, and — the follow-up — why weight decay in the plain adaptive optimiser is not what you meant.

Metrics: not the formulas, but which metric for which situation and why. Why accuracy is useless under imbalance. When precision matters more than recall and when the reverse. What calibration is and why a well-ranked model can be badly calibrated. What a threshold is chosen from.

And a set of questions that are really about honesty: how do you know your model works; what would make you not deploy it; how would you tell if it degraded; what is your model bad at. The right answers here sound like the limitations section from the portfolio topic, and a candidate who answers them concretely is distinguishable in a sentence from one who has only read.

Two habits for the whole topic. Practise aloud, because these rounds are performances and thinking silently then presenting a finished answer is a worse performance than narrating a reasonable one. And say what you are unsure about rather than bluffing — "I would look up the exact argument, but the shape is this" is a strong answer, and a confident wrong detail is the one thing that reliably damages an otherwise good round.

What you should now be able to explain or do

Say what programming level is actually asked and what is rarely asked, and stop practising the wrong end. Perform the recurring dataframe and query operations under time pressure, especially ranking within a group and window functions. Implement k-means, logistic regression and attention from memory, and answer the standard follow-ups including why attention divides by the square root of the dimension. Diagnose bias against variance from a symptom and give the different responses. Choose metrics by situation and explain calibration and thresholds. Answer the honesty questions concretely. Practise aloud and flag uncertainty rather than bluffing.

Check yourself