4.4 Task environments and the four agent types

Work through these

• Classify an environment as observable or partially observable, deterministic or stochastic, episodic or sequential

  Whether the agent can see the whole state, whether actions have certain effects, and whether each episode stands alone. These three dimensions decide most of what an agent needs to be able to do.

• Classify it as static or dynamic, discrete or continuous, single-agent or multi-agent

  Whether the world changes while the agent thinks, whether states and time come in steps or continuously, and whether other agents are present. The last one is what turns a puzzle into a game.

• Classify real tasks against all six dimensions at once

  Applying all six dimensions to real tasks at once, which is where the classification stops being an exercise. Most real environments sit awkwardly between the clean categories.

• Compare simple reflex, model-based reflex, goal-based and utility-based agents

  Four agent structures of increasing sophistication: reacting to the current percept, keeping an internal model, working toward a goal, and weighing outcomes by preference. Each adds one capability the previous lacked.

• Say which agent type a given environment forces, and why

  The environment decides the minimum agent structure required, and building less than that means the agent cannot succeed. Being able to argue this is the point of the whole classification.