Biology subjects

Nelson, J. D.

Publications and source records attributed to Nelson, J. D..

3 recordsLinked to original sources

bioRxiv2019

Exploring the space of human exploration

What drives peoples exploration in complex scenarios where they have to actively acquire information by making queries? How do people adapt their selection of queries to their environment? We explore these questions using Entropy Mastermind, a novel variant of the Mastermind code-breaking game, in which participants have to guess a secret code by making useful queries. Participants solved games more efficiently and more quickly if the entropy of the game environment was low; moreover, people adapted their initial queries to the scenario they were in. We also investigated whether it would be possible to predict participants queries within the generalized Sharma-Mittal information-theoretic framework. Although predicting individual queries is difficult, the modeling framework offered important insight on human behavior. Entropy Mastermind offers rich possibilities for modeling and behavioral research.

animal behavior and cognition↗

bioRxiv2017

Exploration and generalization in vast spaces

From foraging for food to learning complex games, many aspects of human behaviour can be framed as a search problem with a vast space of possible actions. Under finite search horizons, optimal solutions are generally unobtainable. Yet how do humans navigate vast problem spaces, which require intelligent exploration of unobserved actions? Using a variety of bandit tasks with up to 121 arms, we study how humans search for rewards under limited search horizons, where the spatial correlation of rewards (in both generated and natural environments) provides traction for generalization. Across a variety of diifferent probabilistic and heuristic models, we find evidence that Gaussian Process function learning--combined with an optimistic Upper Confidence Bound sampling strategy--provides a robust account of how people use generalization to guide search. Our modelling results and parameter estimates are recoverable, and can be used to simulate human-like performance, providing insights about human behaviour in complex environments.

animal behavior and cognition↗

bioRxiv2017

Mapping the unknown: The spatially correlated multi-armed bandit

We introduce the spatially correlated multi-armed bandit as a task coupling function learning with the exploration-exploitation trade-off. Participants interacted with bi-variate reward functions on a two-dimensional grid, with the goal of either gaining the largest average score or finding the largest payoff. By providing an opportunity to learn the underlying reward function through spatial correlations, we model to what extent people form beliefs about unexplored payoffs and how that guides search behavior. Participants adapted to assigned payoff conditions, performed better in smooth than in rough environments, and--surprisingly--sometimes performed equally well in short as in long search horizons. Our modeling results indicate a preference for local search options, which when accounted for, still suggests participants were best-described as forming local inferences about unexplored regions, combined with a search strategy that directly traded off between exploiting high expected rewards and exploring to reduce uncertainty about the spatial structure of rewards.

animal behavior and cognition↗