Object-Oriented Reinforcement Learning in Cooperative Multiagent Domains

Abstract

Although Reinforcement Learning methods have successfully been applied to increasingly large problems, scalability remains a central issue. While Object-Oriented Markov Decision Processes (OO-MDP) are used to exploit regularities in a domain, Multiagent System (MAS) methods are used to divide workload amongst multiple agents. In this work we propose a novel combination of OO-MDP and MAS, called Multiagent Object-Oriented Markov Decision Process (MOO-MDP), so as to accrue the benefits of both strategies and be able to better address scalability issues. We present an algorithm to solve deterministic cooperative MOO-MDPs, and prove that it learns optimal policies while reducing the learning space by exploiting state abstractions. We experimentally compare our results with earlier approaches and show advantages with regard to discounted cumulative reward, number of steps to fulfill the task, and Q-table size.