S5-3.4 Finite Word Length Effects & Multirate DSP

What this is and why it exists

Everything so far assumed exact arithmetic. A real processor has a fixed number of bits, and a filter that was perfect on paper starts to misbehave.

The second half of this topic is the practical counterweight. Changing the sample rate on purpose is one of the largest savings available in signal processing, and it is used everywhere.

The vocabulary

• Truncation — dropping the extra bits of a result.
• Rounding — choosing the nearest representable value instead.
• Coefficient quantization — storing the filter coefficients in finite precision, so the filter built is not quite the filter designed.
• Limit cycle — a small oscillation that continues forever with no input, caused purely by rounding.
• Overflow oscillation — a large oscillation caused by sums exceeding what the register holds.
• Saturation arithmetic — clamping an overflowing sum to the largest representable value.
• Wraparound — letting an overflowing sum roll over to the opposite sign.
• Decimation — lowering the sample rate by keeping one sample in every D.
• Interpolation — raising the sample rate by inserting samples and filling them in.
• Noble identities — rules that let a filter and a rate change swap places.
• Multistage decimation — reaching a large rate change through several small ones.

The mental model

Three separate errors arrive with finite precision, and they are worth keeping apart.

Round-off and truncation errors appear in the results of arithmetic. Every multiplication produces more bits than it started with, and those extra bits have to go. Rounding is better behaved than truncation because its error has no bias.

Coefficient quantization is different. It happens once, before the filter ever runs. The coefficients you computed cannot be stored exactly, so the poles and zeros move slightly. A sharp, high-order filter has poles crowded near the unit circle, and a small move can push one outside it. That is a stable design becoming an unstable filter, and it is a strong argument for cascading second order sections.

Limit cycles are the memorable case. Take a stable filter, remove the input entirely, and it can still oscillate forever with a small steady pattern. Nothing in the continuous analysis predicts this. It happens because rounding makes the arithmetic slightly nonlinear, and the feedback keeps the rounding error alive.

Overflow oscillations are the violent version. A sum exceeds the register, and what happens next is a design decision. Wraparound flips a large positive number to a large negative one, which the feedback can amplify into a full-scale oscillation. Saturation clamps instead, which distorts but does not explode. The two failures look nothing alike.

Now the multirate half. Decimation lowers the rate by keeping one sample in every D. Do it directly and you get aliasing, because the signal still contains frequencies too high for the new rate. So the filter comes first and the discarding second. That order is the thing people get wrong.

Interpolation is the mirror. Insert zeros between samples to raise the rate, then filter to replace those zeros with sensible values. Combining the two gives any rational rate change. The order matters again: interpolate first, then decimate, or you throw away detail you were about to need.

The noble identities look like algebra and are the key to efficiency. They say that filtering then decimating can be rearranged into decimating then filtering a related filter, and similarly for interpolation. That lets the expensive filtering happen at the lower rate.

Multistage decimation is where all of it pays. A large rate change done in one step needs a filter with a very narrow transition, which means a very high order. Split the change into stages and each stage needs only a gentle filter. The total work drops sharply. Building both versions and comparing the cost is what makes the saving concrete.

What you should now be able to explain or do

Separate round-off, truncation and coefficient quantization, and say when each occurs. Explain why coefficient quantization can destabilise a sharp filter, and why cascaded second order sections help. Describe a limit cycle and say why linear analysis cannot predict it. Compare saturation with wraparound on overflow. Order the operations correctly for decimation, interpolation and rational rate change, and explain why multistage decimation is cheaper.

Check yourself