Worlds

Every AFABL module and agent is designed to operate in a world. A world defines the states, actions and state transition dynamics for particular state and action types.

The states of a world can be represented with any kind of Scala class. Case classes are good for representing states because of their concise syntax and built-in equality methods. Here is a case class for a state with five state variables: the locations of Pac Man, the food pellets, and the ghost.

case class Location(x: Int, y: Int)

case class PacManState(
  pacMan: Location,
  ghost: Location,
  food: Seq[Location],
  cherries: Seq[Location],
  cherryActive: Boolean
)

Actions are also represented by objects which can be instances of any class. As with states, case classes make a good choice for implementing actions. Here we show actions for the Pac Man agent implemented as a Scala enumeration.

object PacManAction extends Enumeration {
  type PacManAction = Value
  val Up, Down, Left, Right = Value
}

World Dynamics

An agent executes actions in a world, and those actions potentially change the state of the world. Having defined Scala representations for states and actions, we can define a world. Below is the abstract class which defines the basic interface of world objects, which are instances of subclasses if World. All modules and agents are defined to act in a particular instance of a world. As with states and actions, world representations make no advanced use of the Scala programming language.

abstract class World[S, A] {
  def init(): S
  def resetAgent(): S
  def states: Seq[S]
  def actions: Seq[A]
  def act(action: A): S
}

Here are the important portions of the Pac Man world with explanatory comments.

class PacManWorld extends World[PacManState, PacManAction] {

  def init(): S = {
    // Put PacMan, and the ghost in their start states,
    // place all the food pellets and cherries
  }

  def resetAgent(): S = {
    // "Respawn" the PacMan back in its start state
  }

  // In Scala you can override a def with a val
  val states: Seq[S] = {
    // All the possible states of the PacManWorld
    // Needed by reinforcement learning algorithms
    // Yes, this is huge, but it's computed only once.
  }

  def actions: Seq[A] = {
    // All the possible actions PacMan can execute in the PacManWorld
    // Needed by reinforcement learning algorithms
  }
  def act(action: A): S = {
    // Execute PacMan's action, update the state of the world including
    // PacMan's movement, the ghost's movement, disappearing food pellets,
    // and possibly cherryActive. Return the updated state.
  }
}