Coordinating decentralized learning and conflict resolution across agent boundaries

1 online resource (178 pages) : PDF

2012

University of North Carolina at Charlotte

It is crucial for embedded systems to adapt to the dynamics of openenvironments. This adaptation process becomes especially challenging in the context of multiagent systems because of scalability, partial information accessibility and complex interaction of agents. It is a challenge for agents to learn good policies, when they need to plan and coordinate in uncertain, dynamic environments, especially when they have large state spaces. It is also critical for agents operating in a multiagent system (MAS) to resolve conflicts among the learned policies of different agents, since such conflicts may have detrimental influence on the overall performance.The focus of this research is to use a reinforcement learning based local optimization algorithm within each agent to learn multiagent policies in a decentralized fashion. These policies will allow each agent to adapt to changes in environmental conditions while reorganizing the underlying multiagent network when needed. The research takes an adaptive approach to resolving conflicts that can arise between locally optimal agent policies. First an algorithm that uses heuristic rules to locally resolve simple conflicts is presented. When the environment is more dynamic and uncertain, a mediator-based mechanism to resolve more complicated conflicts and selectively expand the agents' state space during the learning process is harnessed. For scenarios where mediator-based mechanisms with partially global views are ineffective, a more rigorous approach for global conflict resolution that synthesizes multiagent reinforcement learning (MARL) and distributed constraint optimization (DCOP) is developed. These mechanisms are evaluated in the context of a multiagent tornado tracking application called NetRads. Empirical results show that these mechanismssignificantly improve the performance of the tornado trackingnetwork for a variety of weather scenarios.The major contributions of this work are: a state of the art decentralized learning approach that supports agent interactions and reorganizes the underlying network when needed; the use of abstract classes of scenarios/states/actions that efficiently manages the exploration of the search space; novel conflict resolution algorithms of increasing complexity that use heuristic rules, sophisticated automated negotiation mechanisms and distributed constraint optimization methods respectively; and finally, a rigorous study of the interplay between two popular theories used to solve multiagent problems, namely decentralized Markov decision processes and distributed constraint optimization.

doctoral dissertations

Computer science
Information technology
Artificial intelligence

Ph.D.

Conflict Resolution
DEC-MDPS
Distributed Constraint Optimization
Meta-Level Control
Multiagent Systems
Reinforcement Learning

Information Technology

Raja, Anita

Raja, Anita
Xiao, Jing
Hadzikadic, Mirsad
Xie, Jiang (Linda)
Brown, Mary

Thesis (Ph.D.)--University of North Carolina at Charlotte, 2012.

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). For additional information, see http://rightsstatements.org/page/InC/1.0/.

Copyright is held by the author unless otherwise indicated.

Cheng_uncc_0694D_10387

http://hdl.handle.net/20.500.13093/etd:754