Intelligent Systems
Note: This research group has relocated.

10 results (View BibTeX file of all listed publications)

2022

no image
Can we improve self-regulation during computer-based work with optimal feedback?

Wirzberger, M., Lado, A., Prentice, M., Oreshnikov, I., Passy, J., Stock, A., Lieder, F.

Behaviour & Information Technology, November 2022 (article) Submitted

Abstract
Distractions are omnipresent and can derail our attention, which is a precious and very limited resource. To achieve their goals in the face of distractions, people need to regulate their attention, thoughts, and behavior; this is known as self-regulation. How can self-regulation be supported or strengthened in ways that are relevant for everyday work and learning activities? To address this question, we introduce and evaluate a desktop application that helps people stay focused on their work and train self-regulation at the same time. Our application lets the user set a goal for what they want to do during a defined period of focused work at their computer, then gives negative feedback when they get distracted, and positive feedback when they reorient their attention towards their goal. After this so-called focus session, the user receives overall feedback on how well they focused on their goal relative to previous sessions. While existing approaches to attention training often use artificial tasks, our approach transforms real-life challenges into opportunities for building strong attention control skills. Our results indicate that optimal attentional feedback can generate large increases in behavioral focus, task motivation, and self-control – benefitting users to successfully achieve their long-term goals.

link (url) [BibTex]

An interdisciplinary synthesis of research on understanding and promoting well-doing
An interdisciplinary synthesis of research on understanding and promoting well-doing

Lieder, F., Prentice, M., Corwin-Renner, E.

Social and Personality Psychology Compass, 16(9), September 2022 (article)

Abstract
People’s intentional pursuit of prosocial goals and values (i.e., well-doing) is critical to the flourishing of humanity in the long run. Understanding and promoting well-doing is a shared goal across many fields inside and outside of social and personality psychology. Several of these fields are (partially) disconnected from each other and could benefit from more integration of existing knowledge, interdisciplinary collaboration, and cross-fertilization. To foster the transfer and integration of knowledge across these different fields, we provide a brief overview with pointers to some of the key articles in each field, highlight connections, and introduce an integrative model of the psychological mechanisms of well-doing. We identify some gaps in the current understanding of well-doing, such as the paucity of research on well-doing with large and long-lasting positive consequences. Building on this analysis, we identify opportunities for high-impact research on well-doing in social and personality psychology, such as understanding and promoting the effective pursuit of highly impactful altruistic goals.

link (url) DOI [BibTex]

Lieder, F., Prentice, M., Corwin-Renner, E. An interdisciplinary synthesis of research on understanding and promoting well-doing Social and Personality Psychology Compass, 16(9), September 2022 (article)

link (url) DOI [BibTex]

Boosting human decision-making with AI-generated decision aids
Boosting human decision-making with AI-generated decision aids

Becker, F., Skirzyński, J., van Opheusden, B., Lieder, F.

Computational Brain & Behavior, 5(4):467-490, July 2022 (article)

Abstract
Human decision-making is plagued by many systematic errors. Many of these errors can be avoided by providing decision aids that guide decision-makers to attend to the important information and integrate it according to a rational decision strategy. Designing such decision aids is a tedious manual process. Advances in cognitive science might make it possible to automate this process in the future. We recently introduced machine learning methods for discovering optimal strategies for human decision-making automatically and an automatic method for explaining those strategies to people. Decision aids constructed by this method were able to improve human decision-making. However, following the descriptions generated by this method is very tedious. We hypothesized that this problem can be overcome by conveying the automatically discovered decision strategy as a series of natural language instructions for how to reach a decision. Experiment 1 showed that people do indeed understand such procedural instructions more easily than the decision aids generated by our previous method. Encouraged by this finding, we developed an algorithm for translating the output of our previous method into procedural instructions. We applied the improved method to automatically generate decision aids for a naturalistic planning task (i.e., planning a road trip) and a naturalistic decision task (i.e., choosing a mortgage). Experiment 2 showed that these automatically generated decision-aids significantly improved people's performance in planning a road trip and choosing a mortgage. These findings suggest that AI-powered boosting has potential for improving human decision-making in the real world.

DOI [BibTex]

Becker, F., Skirzyński, J., van Opheusden, B., Lieder, F. Boosting human decision-making with AI-generated decision aids Computational Brain & Behavior, 5(4):467-490, July 2022 (article)

DOI [BibTex]

no image
Leveraging machine learning to automatically derive robust decision strategies from imperfect models of the real world

Mehta, A., Jain, Y. R., Kemtur, A., Stojcheski, J., Consul, S., Tosic, M., Lieder, F.

Computational Brain & Behavior, 5, pages: 343-377, Springer Nature, June 2022 (article)

Abstract
Teaching people clever heuristics is a promising approach to improve decision-making under uncertainty. The theory of resource-rationality makes it possible to leverage machine learning to discover optimal heuristics automatically. One bottleneck of this approach is that the resulting decision strategies are only as good as the model of the decision-problem that the machine learning methods were applied to. This is problematic because even domain experts cannot give complete and fully accurate descriptions of the decisions they face. To address this problem, we develop strategy discovery methods that are robust to potential inaccuracies in the description of the scenarios in which people will use the discovered decision strategies. The basic idea is to derive the strategy that will perform best in expectation across all possible real-world problems that could have given rise to the likely erroneous description that a domain expert provided. To achieve this, our method uses a probabilistic model of how the description of a decision problem might be corrupted by biases in human judgment and memory. Our method uses this model to perform Bayesian inference on which real-world scenarios might have given rise to the provided descriptions. We applied our Bayesian approach to robust strategy discovery in two domains: planning and risky choice. In both applications, we find that our approach is more robust to errors in the description of the decision problem and that teaching the strategies it discovers significantly improves human decision-making in scenarios where approaches ignoring the risk that the description might be incorrect are ineffective or even harmful. The methods developed in this article are an important step towards leveraging machine learning to improve human decision-making in the real world because they tackle the problem that the real world is fundamentally uncertain.

Leveraging Machine Learning to Automatically Derive Robust Decision Strategies from Imperfect Knowledge of the Real World link (url) DOI [BibTex]

Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning
Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning

Consul, S., Heindrich, L., Stojcheski, J., Lieder, F.

Computational Brain & Behavior, 5, pages: 185-216, Springer, 2022 (article)

Abstract
To make good decisions in the real world people need efficient planning strategies because their computational resources are limited. Knowing which planning strategies would work best for people in different situations would be very useful for understanding and improving human decision-making. But our ability to compute those strategies used to be limited to very small and very simple planning tasks. To overcome this computational bottleneck, we introduce a cognitively-inspired reinforcement learning method that can overcome this limitation by exploiting the hierarchical structure of human behavior. The basic idea is to decompose sequential decision problems into two sub-problems: setting a goal and planning how to achieve it. This hierarchical decomposition enables us to discover optimal strategies for human planning in larger and more complex tasks than was previously possible. The discovered strategies outperform existing planning algorithms and achieve a super-human level of computational efficiency. We demonstrate that teaching people to use those strategies significantly improves their performance in sequential decision-making tasks that require planning up to eight steps ahead. By contrast, none of the previous approaches was able to improve human performance on these problems. These findings suggest that our cognitively-informed approach makes it possible to leverage reinforcement learning to improve human decision-making in complex sequential decision-problems. Future work can leverage our method to develop decision support systems that improve human decision making in the real world.

link (url) DOI Project Page Project Page [BibTex]

no image
Rational use of cognitive resources in human planning

Callaway, F., Opheusden, B. V., Gul, S., Das, P., Krueger, P. M., Griffiths, T. L., Lieder, F.

Nature Human Behaviour, 6, pages: 1112-1125, April 2022 (article)

Abstract
Making good decisions requires thinking ahead, but the huge number of actions and outcomes one could consider makes exhaustive planning infeasible for computationally constrained agents, such as humans. How people are nevertheless able to solve novel problems when their actions have long-reaching consequences is thus a long-standing question in cognitive science. To address this question, we propose a model of resource-constrained planning that allows us to derive optimal planning strategies. We find that previously proposed heuristics such as best-first search are near-optimal under some circumstances, but not others. In a mouse-tracking paradigm, we show that people adapt their planning strategies accordingly, planning in a manner that is broadly consistent with the optimal model but not with any single heuristic model. We also find systematic deviations from the optimal model that might result from additional cognitive constraints that are yet to be uncovered.

link (url) DOI [BibTex]

Callaway, F., Opheusden, B. V., Gul, S., Das, P., Krueger, P. M., Griffiths, T. L., Lieder, F. Rational use of cognitive resources in human planning Nature Human Behaviour, 6, pages: 1112-1125, April 2022 (article)

link (url) DOI [BibTex]

What to learn next? Aligning gamification rewards to long-term goals using reinforcement learning
What to learn next? Aligning gamification rewards to long-term goals using reinforcement learning

Pauly, R., Heindrich, L., Amo, V., Lieder, F.

March 2022 (article) Accepted

Abstract
Nowadays, more people can access digital educational resources than ever before. However, access alone is often not sufficient for learners to fulfil their learning goals. To support motivation, learning environments are often gamified, meaning that they offer points for interacting with them. But gamification can add to learners’ tendencies to choose learning activities in a short-sighted manner. An example for a short-sighted choice bias is the preference for an easy task offering a quick sense of accomplishment (and in gamified environments often a quick accumulation of points) over a harder task offering to make real progress. The concept of optimal brain points demonstrates that methods from the field of reinforcement learning, specifically reward shaping, allow us to align short-term rewards for learning choices with their expected long-term benefit in a learning context. Building on that work, we here present a scalable approach to supporting self-directed learning in digital learning environments applicable to real-world educational games. It can motivate learners to choose the learning activities that are most beneficial for them in the long run. This is achieved by incentivizing each learning activity in a way that reflects how much progress can be made by completing it and how that progress relates to their learning goal. Specifically, the approach entails modelling how learners choose between learning activities as a Markov Decision Process and applying methods from reinforcement learning to compute which learning choices optimize the learners progress based on their current knowledge. We specify how our developed method can be applied to the English-learning App “Dawn of Civilisation”. We further present the first evaluation of the approach in a controlled online experiment with a simplified learning task, which showed that the derived incentives can significantly improve both learners’ choice behaviour and their learning outcomes.

link (url) [BibTex]

Leveraging artificial intelligence to improve people’s planning strategies
Leveraging artificial intelligence to improve people’s planning strategies

Callaway, F., Jain, Y. R., Opheusden, B. V., Das, P., Iwama, G., Gul, S., Krueger, P. M., Becker, F., Griffiths, T. L., Lieder, F.

119(12), PNAS, March 2022 (article)

Abstract
Human decision making is plagued by systematic errors that can have devastating consequences. Previous research has found that such errors can be partly prevented by teaching people decision strategies that would allow them to make better choices in specific situations. Three bottlenecks of this approach are our limited knowledge of effective decision strategies, the limited transfer of learning beyond the trained task, and the challenge of efficiently teaching good decision strategies to a large number of people. We introduce a general approach to solving these problems that leverages artificial intelligence to discover and teach optimal decision strategies. As a proof of concept, we developed an intelligent tutor that teaches people the automatically discovered optimal heuristic for environments where immediate rewards do not predict long-term outcomes. We found that practice with our intelligent tutor was more effective than conventional approaches to improving human decision making. The benefits of training with our cognitive tutor transferred to a more challenging task and were retained over time. Our general approach to improving human decision making by developing intelligent tutors also proved successful for another environment with a very different reward structure. These findings suggest that leveraging artificial intelligence to discover and teach optimal cognitive strategies is a promising approach to improving human judgment and decision making.

link (url) DOI [BibTex]

Callaway, F., Jain, Y. R., Opheusden, B. V., Das, P., Iwama, G., Gul, S., Krueger, P. M., Becker, F., Griffiths, T. L., Lieder, F. Leveraging artificial intelligence to improve people’s planning strategies 119(12), PNAS, March 2022 (article)

link (url) DOI [BibTex]

no image
Discovering Rational Heuristics for Risky Choice

Krueger, P., Callaway, F., Gul, S., Griffiths, T., Lieder, F.

PsyArXiv Preprints, January 2022 (article) Submitted

Abstract
For computationally limited agents such as humans, perfectly rational decision-making is almost always out of reach. Instead, people may rely on computationally frugal heuristics that usually yield good outcomes. Although previous research has identified many such heuristics, discovering good heuristics and predicting when they will be used remains challenging. Here, we present a machine learning method that identifies the best heuristics to use in any given situation. To demonstrate the generalizability and accuracy of our method, we compare the strategies it discovers against those used by people across a wide range of multi-alternative risky choice environments in a behavioral experiment that is an order of magnitude larger than any previous experiments of its type. Our method rediscovered known heuristics, identifying them as rational strategies for specific environments, and discovered novel heuristics that had been previously overlooked. Our results show that people adapt their decision strategies to the structure of the environment and generally make good use of their limited cognitive resources, although they tend to collect too little information and their strategy choices do not always fully exploit the structure of the environment.

Discovering Rational Heuristics for Risky Choice link (url) DOI [BibTex]

Krueger, P., Callaway, F., Gul, S., Griffiths, T., Lieder, F. Discovering Rational Heuristics for Risky Choice PsyArXiv Preprints, January 2022 (article) Submitted

Discovering Rational Heuristics for Risky Choice link (url) DOI [BibTex]