Integração de homeostase, aprendizado por reforço e emoções no projeto de personagens virtuais autônomos

Abstract

Game development is a process that involves experimenting with mechanics and scenarios before design decisions can be effectively validated. The advancement in the processing power of modern computers, along with the emergence of efficient algorithms and approaches for automatic content generation, has significantly expanded the possibilities for creating virtual worlds. Although robust techniques such as navigation meshes (navmesh) and behavior trees (behavior trees) have been developed, the design of autonomous characters still relies heavily on heuristics, often developed in an ad hoc manner and with a significant amount of pre-scripted behavior. This work argues that the development of autonomous virtual characters can benefit from behavior modeling paradigms inspired by biological processes. This work proposes a bio-inspired model that integrates homeostasis, reinforcement learning, and emotions to enhance the behavioral plausibility of characters in three fundamental aspects: increasing behavioral variability, generating emotions consistent with experienced situations, and adapting to environmental changes, either without the need for retraining or with minimal retraining. Additionally, behavioral variability can be controlled to create different character profiles, which is particularly useful for balancing video games. Homeostasis is used as a basis to regulate the internal balance of characters, generating complex behaviors from simulated physiological needs. However, since it is not necessary to directly imitate physiological aspects of biological beings, simulated homeostasis offers greater flexibility, allowing the inclusion of variables not necessarily related to the concept of biological physiology but that the agent must maintain in balance, such as minimum and maximum distance from other characters. Reinforcement learning enables characters to adjust their actions based on previous experiences and the reinforcement generated by seeking homeostatic balance. Finally, emotions modulate the interaction between homeostasis and learning, increasing the explainability of character behaviors, which facilitates the acceptance and understanding of these behaviors by human participants in virtual worlds. This work demonstrates how these components can be combined to construct a coherent model of an autonomous virtual character.