Design of a learning system

The design of a learning system is a process of constructing a system that uses a learning algorithm to improve its performance on a given task based on experience. It involves four major steps:

1. Choosing a Training Experience

• Training experience refers to the data or interactions through which the system learns.
• It can be of two types:
  • Direct Experience: The system (e.g., a chess-playing agent) is provided with board positions and correct moves explicitly.
  • Indirect Experience: The system plays against itself or an opponent and learns from the outcomes.
• The presence of a supervisor can help label data (like valid moves).
• If the training and testing samples share similar distributions, the system performance improves.

2. Choosing a Target Function

• The target function represents what the system is trying to learn.
• In a chess game example, the function could evaluate if a move is good:
  • For example:
    B > M means move B is better than move M (both are legal moves).
• If all legal moves are scored, the one with the highest score is selected.

3. Representation of the Target Function

• The target function is represented using a hypothesis, such as:

• This is a linear representation but can be extended to rules, tables, or neural networks.

4. Function Approximation

• The approximation aims to find the best hypothesis that matches the target function.

Components of a Learning System

1. Performance System – Lets the system (e.g., game agent) interact with the environment (e.g., plays chess).
2. Critic System – Evaluates the performance and provides feedback (error).
3. Generalizer System – Learns a general hypothesis from the feedback.
4. Experimenter System – Proposes new strategies based on learned experience and sends it to the performance system for further testing.