Intelligent Systems
Note: This research group has relocated.

57 results (View BibTeX file of all listed publications)
no image
Toward a normative theory of (self-)management by goal-setting

Singhi, N., Mohnert, F., Prystawski, B., Lieder, F.

Proceedings of the Annual Meeting of the Cognitive Science Society, Annual Meeting of the Cognitive Science Society, July 2023 (conference) Accepted

link (url) DOI [BibTex]

Singhi, N., Mohnert, F., Prystawski, B., Lieder, F. Toward a normative theory of (self-)management by goal-setting Proceedings of the Annual Meeting of the Cognitive Science Society, Annual Meeting of the Cognitive Science Society, July 2023 (conference) Accepted

link (url) DOI [BibTex]

2022

no image
Which research topics are most important for promoting flourishing?

Lieder, F.

In Global Conference on Human Flourishing, Templeton World Charity Foundation, November 2022 (inproceedings) Accepted

link (url) DOI [BibTex]

2022

Lieder, F. Which research topics are most important for promoting flourishing? In Global Conference on Human Flourishing, Templeton World Charity Foundation, November 2022 (inproceedings) Accepted

link (url) DOI [BibTex]

no image
A cautionary tale about AI-generated goal suggestions

Lieder, F., Chen, P., Stojcheski, J., Consul, S., Pammer-Schindler, V.

In MuC ’22: Proceedings of Mensch und Computer 2022 , pages: 354-359, Mensch und Computer 2022 (MuC 2022) , September 2022 (inproceedings)

Abstract
Setting the right goals and prioritizing them might be the most crucial and the most challenging type of decisions people make for themselves, their teams, and their organizations. In this article, we explore whether it might be possible to leverage artificial intelligence (AI) to help people set better goals and which potential problems might arise from such applications. We devised the first prototype of an AI-powered digital goal-setting assistant and a rigorous empirical paradigm for assessing the quality of AI-generated goal suggestions. Our empirical paradigm compares the AI-generated goal suggestions against randomly-generated goal suggestions and unassisted goal-setting on a battery of self-report measures of important goal characteristics, motivation, and usability in a large-scale repeated-measures online experiment. The results of an online experiment with 259 participants revealed that our intuitively compelling goal suggestion algorithm was actively harmful to the quality of the people's goals and their motivation to pursue them. These surprising findings highlight three crucial problems to be tackled by future work on leveraging AI to help people set better goals: i) aligning the objective function of the AI algorithms with the design goals, ii) helping people quantify how valuable different goals are to them, and iii) preserving the user's sense of autonomy.

link (url) DOI [BibTex]

Lieder, F., Chen, P., Stojcheski, J., Consul, S., Pammer-Schindler, V. A cautionary tale about AI-generated goal suggestions In MuC ’22: Proceedings of Mensch und Computer 2022 , pages: 354-359, Mensch und Computer 2022 (MuC 2022) , September 2022 (inproceedings)

link (url) DOI [BibTex]

no image
Leveraging AI for effective to-do list gamification

Consul, S., Stojcheski, J., Lieder, F.

In Mensch und Computer 2022 – Workshopband MuC 2022 , Mensch und Computer 2022 (MuC 2022) : 5th International Workshop "Gam-R – Gamification Reloaded" , September 2022 (inproceedings)

link (url) DOI [BibTex]

Consul, S., Stojcheski, J., Lieder, F. Leveraging AI for effective to-do list gamification In Mensch und Computer 2022 – Workshopband MuC 2022 , Mensch und Computer 2022 (MuC 2022) : 5th International Workshop "Gam-R – Gamification Reloaded" , September 2022 (inproceedings)

link (url) DOI [BibTex]

no image
Does deliberate prospection help students set better goals?

Jähnichen, S., Weber, F., Prentice, M., Lieder, F.

In 15th Biannual Meeting of the German Cognitive Science Society , pages: 188-189 , 15th Biannual Meeting of the German Cognitive Science Society (KogWis 2022 – Understanding Minds) , September 2022 (inproceedings)

link (url) [BibTex]

Jähnichen, S., Weber, F., Prentice, M., Lieder, F. Does deliberate prospection help students set better goals? In 15th Biannual Meeting of the German Cognitive Science Society , pages: 188-189 , 15th Biannual Meeting of the German Cognitive Science Society (KogWis 2022 – Understanding Minds) , September 2022 (inproceedings)

link (url) [BibTex]

no image
Promoting value-congruent action by supporting effective metacognitive emotion-regulation strategies with a gamified app

Amo, V., Prentice, M., Lieder, F.

Society for Personality and Social Psychology (SPSP) Annual Convention 2022, San Francisco, USA, Society for Personality and Social Psychology (SPSP) Annual Convention 2022, February 2022 (conference)

Abstract
Negative emotions can make maladaptive behavior more likely, especially when people have poor emotion regulation and metacognitive skills (ERMSs). We developed an app to help non-clinical populations train and apply good ERMSs. The app teaches ERMSs with the help of gamified features such as customizable emotion avatars and points for practicing ERMSs. In an initial, brief pre/post test of the app, 60 participants used it to reflect on a difficult emotional challenge and (non-)beneficial ways of acting. Then, they completed a metacognitive skill-building module. After using the app, participants' scores showed significantly decreased/increased perceived likelihood of unwanted/beneficial actions, decreased emotional struggle and emotional intensity, and decreased/increased cognitive endorsement of self-limiting/self-efficacious beliefs (Paired Samples Wilcoxon Test average effect size = 0.71, range = [.26, .87], all p<0.008). These results provide an important proof-of-concept for the app. A subsequent study will test the app's effectiveness for at least two weeks using event-contingent reporting for participants' real-life regulatory challenges and ERMS training in context.

[BibTex]

Amo, V., Prentice, M., Lieder, F. Promoting value-congruent action by supporting effective metacognitive emotion-regulation strategies with a gamified app Society for Personality and Social Psychology (SPSP) Annual Convention 2022, San Francisco, USA, Society for Personality and Social Psychology (SPSP) Annual Convention 2022, February 2022 (conference)

[BibTex]

no image
Evaluating Life Reflection Techniques to Help People Select Virtuous Life Goals

Prentice, M., Gonzalez Cruz, H., Lieder, F.

Integrating Research on Character and Virtues: 10 Years of Impact, Oriel College, Oxford, Integrating Research on Character and Virtues: 10 Years of Impact, January 2022 (conference) Accepted

Abstract
The purpose of the present studies was to identify an effective tool for helping people to select virtuous life goals that promote their own well-being and contribute to the well-being of others (well-doing). Across two studies, we tested four candidate interventions against each other and a control condition. In the first study (N = 218), the intervention conditions were the eulogy and valued living questionnaire exercises from the Acceptance and Commitment Therapy literature. In the second study (N = 537), the intervention conditions were self-affirmation and value self-confrontation from the social psychology literature and the eulogy exercise. The eulogy exercise is a very brief reflection (3-5 minutes) on how one would like to be remembered by friends and family speaking at one’s funeral. The valued living questionnaire exercise involves rating 10 life domains for importance and behavioral consistency with that importance and reflecting on discrepancies. Self-affirmation involves writing about a time when one acted in line with one’s values. And value self-confrontation involves inducing a discrepancy between participants’ values and those of a socially desirable group. Participants were randomly assigned to one of these brief interventions or a control condition. They were then asked to select a life goal that they would like to start pursuing or make more progress on in the near future. In Study 1, selection was open-ended and participants indicated which of 5 life domains it best fit, including interpersonal goals. In Study 2, the goal was selected from a list of prosocial, personal growth, or materialistic life goals. Across both studies, we found that the eulogy exercise stood out as an effective intervention for helping people select life goals that are likely to promote well-being and well-doing, such as wanting to improve other people’s lives, and avoid life goals that are associated with vices, such as wanting to have many expensive possessions. These findings point to the usefulness of humanistic-existential approaches for promoting character development via life goals and provide an example of how philosophically-informed psychological interventions can be effective.

[BibTex]

Prentice, M., Gonzalez Cruz, H., Lieder, F. Evaluating Life Reflection Techniques to Help People Select Virtuous Life Goals Integrating Research on Character and Virtues: 10 Years of Impact, Oriel College, Oxford, Integrating Research on Character and Virtues: 10 Years of Impact, January 2022 (conference) Accepted

[BibTex]

2021

Promoting metacognitive learning through systematic reflection
Promoting metacognitive learning through systematic reflection

Becker, F., Lieder, F.

Workshop on Metacognition in the Age of AI. Thirty-fifth Conference on Neural Information Processing Systems, 35th Conference on Neural Information Processing Systems (NeurIPS 2021), December 2021 (conference)

Abstract
People are able to learn clever cognitive strategies through trial and error from small amounts of experience. This is facilitated by people's ability to reflect on their own thinking which is known as metacognition. To examine the effects of deliberate systematic metacognitive reflection on how people learn how to plan, the experimental group was guided to systematically reflect on their decision-making process after every third decision. We found that participants assisted by reflection prompts learned to plan better faster. Moreover, we found that reflection led to immediate improvements in the participants' planning strategies. Our preliminary results do suggest that deliberate metacognitive reflection can help people discover clever cognitive strategies from very small amounts of experience. Understanding the role of reflection in human learning is a promising approach for making reinforcement learning more sample efficient in both humans and machines.

link (url) DOI Project Page [BibTex]

2021

Becker, F., Lieder, F. Promoting metacognitive learning through systematic reflection Workshop on Metacognition in the Age of AI. Thirty-fifth Conference on Neural Information Processing Systems, 35th Conference on Neural Information Processing Systems (NeurIPS 2021), December 2021 (conference)

link (url) DOI Project Page [BibTex]

Have I done enough planning or should I plan more?
Have I done enough planning or should I plan more?

He, R., Jain, Y. R., Lieder, F.

Workshop on Metacognition in the Age of AI. Thirty-fifth Conference on Neural Information Processing Systems, Long Paper, Workshop on Metacognition in the Age of AI. Thirty-fifth Conference on Neural Information Processing Systems, December 2021 (conference) Accepted

Abstract
People’s decisions about how to allocate their limited computational resources are essential to human intelligence. An important component of this metacognitive ability is deciding whether to continue thinking about what to do and move on to the next decision. Here, we show that people acquire this ability through learning and reverse-engineer the underlying learning mechanisms. Using a process-tracing paradigm that externalises human planning, we find that people quickly adapt how much planning they perform to the cost and benefit of planning. To discover the underlying metacognitive learning mechanisms we augmented a set of reinforcement learning models with metacognitive features and performed Bayesian model selection. Our results suggest that the metacognitive ability to adjust the amount of planning might be learned through a policy-gradient mechanism that is guided by metacognitive pseudo-rewards that communicate the value of planning.

Project Page [BibTex]

He, R., Jain, Y. R., Lieder, F. Have I done enough planning or should I plan more? Workshop on Metacognition in the Age of AI. Thirty-fifth Conference on Neural Information Processing Systems, Long Paper, Workshop on Metacognition in the Age of AI. Thirty-fifth Conference on Neural Information Processing Systems, December 2021 (conference) Accepted

Project Page [BibTex]

Encouraging far-sightedness with automatically generated descriptions of optimal planning strategies: Potentials and Limitations
Encouraging far-sightedness with automatically generated descriptions of optimal planning strategies: Potentials and Limitations

Becker, F., Skirzynski, J. M., van Opheusden, B., Lieder, F.

Proceedings of the 43rd Annual Meeting of the Cognitive Science Society, Online, Annual Meeting of the Cognitive Science Society, July 2021 (conference)

Abstract
People often fall victim to decision-making biases, e.g. short-sightedness, that lead to unfavorable outcomes in their lives. It is possible to overcome these biases by teaching people better decision-making strategies. Finding effective interventions is an open problem, with a key challenge being the lack of transfer to the real world. Here, we tested a new approach to improving human decision-making that leverages Artificial Intelligence to discover procedural descriptions of effective planning strategies. Our benchmark problem regarded improving far-sightedness. We found our intervention elicits transfer to a similar task in a different domain, but its effects in more naturalistic financial decisions were not statistically significant. Even though the tested intervention is on par with conventional approaches, which also struggle in far-transfer, further improvements are required to help people make better decisions in real life. We conclude that future work should focus on training decision-making in more naturalistic scenarios.

link (url) [BibTex]

Becker, F., Skirzynski, J. M., van Opheusden, B., Lieder, F. Encouraging far-sightedness with automatically generated descriptions of optimal planning strategies: Potentials and Limitations Proceedings of the 43rd Annual Meeting of the Cognitive Science Society, Online, Annual Meeting of the Cognitive Science Society, July 2021 (conference)

link (url) [BibTex]

no image
Leveraging AI to support the self-directed learning of disadvantaged youth in developing countries

Teo, J., Pauly, R., Heindrich, L., Amo, V., Lieder, F.

The first Life Improvement Science Conference, Tübingen, Germany, The first Life Improvement Science Conference, June 2021 (conference) Accepted

Abstract
Globally 258 million children and youth do not have access to school (Unesco, 2019), while 600 million receive ineffective education (Unesco, 2017). Solve Education! (SE!) is a non-profit organization committed to enable these young people to empower themselves through education, and currently operates in over 7 countries. Their team includes educationists, technologists, and business executives, who work together with governments and local communities to reach young people with disadvantaged backgrounds. Solve Education!’s main mobile application “The Dawn of Civilisation” (DoC), is an open platform that can deliver different learning content, with the focus on English literacy. It is designed to support lower end devices, as well as offline learning. At the Rationality Enhancement Group, we are laying the scientific foundation for helping people do more good in better ways. We combine methods from computational cognitive science, psychology, human-computer interaction, and artificial intelligence for the development of practical tools, strategies, and interventions that support people in their personal growth. In our collaboration with SE!, we aim at learning from and contributing to real-world challenges by applying our research to enhance SE!’s learning platform. We are currently working on two projects. The first project’s goal is to develop a principled approach to incentivize efficient self-directed learning with digital educational resources and to evaluate its effectiveness regarding learners’ behaviors and success in cooperation with SE!. Specifically, SE!’s DoC serves as the digital educational resource and allows to evaluate the approach with very high ecological validity. The planned intervention is based on the concept of optimal brain points developed by Xu, Wirzberger & Lieder (2019). The core idea is to incentivize effort and smart study choices rather than performance and to do so in a way that learners cannot exploit shortcuts to accumulate game points without also moving closer to their actual learning goals. If successful, SE! can build upon the intervention to further enhance the benefits their users draw from DoC. The second project is based on hierarchical goal setting and consists of a digital assistant that helps users set real-world goals and make progress towards them by reaching milestones with DoC. In this talk, in addition to introducing our work together with SE, we will highlight the mutual benefits of the collaboration between scientists and socially impactful organizations.

[BibTex]

Teo, J., Pauly, R., Heindrich, L., Amo, V., Lieder, F. Leveraging AI to support the self-directed learning of disadvantaged youth in developing countries The first Life Improvement Science Conference, Tübingen, Germany, The first Life Improvement Science Conference, June 2021 (conference) Accepted

[BibTex]

Evaluating Life Reflection Techniques to Help People Set Better Value-Driven Life Goals
Evaluating Life Reflection Techniques to Help People Set Better Value-Driven Life Goals

Prentice, M., González Cruz, H., Lieder, F.

13th Annual Conference of the Society for the Science of Motivation, Society for the Science of Motivation, 13th Annual Conference of the Society for the Science of Motivation , May 2021 (conference)

Abstract
We tested two reflection techniques derived from Acceptance Commitment Therapy for helping people set life goals that are self-determined, communal, and future-minded. Participants were assigned randomly to control, Eulogy, or the Valued Living Questionnaire (VLQ) conditions. Eulogy participants envisioned what they wanted people to say about them at their funeral. In VLQ, participants rated the importance of life domains and how consistent their behavior has recently been with the importance assigned to each domain. Participants then set a life goal, rated it for self-determination, and indicated its time horizon and life domain. Despite only requiring internal reflection, Eulogy was particularly effective for generating self-determined goals that were interpersonal and future-minded. The Eulogy exercise may be a useful and important building block for inspiring the setting and effective pursuit of goals that are simultaneously self-determined, communal, and future-minded. Future research will examine its efficacy in changing experienced well-being and enacted well-doing.

[BibTex]

Prentice, M., González Cruz, H., Lieder, F. Evaluating Life Reflection Techniques to Help People Set Better Value-Driven Life Goals 13th Annual Conference of the Society for the Science of Motivation, Society for the Science of Motivation, 13th Annual Conference of the Society for the Science of Motivation , May 2021 (conference)

[BibTex]

'What Do You Want in Life and How Can You Get There?' An Evaluation of a Hierarchical Goal-Setting Chatbot
’What Do You Want in Life and How Can You Get There?’ An Evaluation of a Hierarchical Goal-Setting Chatbot

González Cruz, H., Prentice, M., Lieder, F.

13th Annual meeting of the Society for the Science of Motivation, Abstract of presentation at the 13th SSM Virtual Congress, Society for the Science of Motivation, Virtual Congress, May 2021 (conference)

Abstract
The translation of abstract, long-term goals, such as “make a contribution to the field of motivation science,” into short-term, actionable intentions is inherently difficult. Hierarchical goal-setting, a goal-setting strategy in which people construct a hierarchy of increasingly more concrete and proximal subgoals is a promising way to support this process. We designed a goal-setting chatbot that helps people craft action hierarchies for achieving their life goals. We conducted a large online field experiment with two follow-up surveys at one week and one month after the intervention to evaluate the effects of a brief hierarchical planning session with our chatbot on goal pursuit. Although there were no main effects of hierarchical planning on goal-related outcomes, exploratory analyses indicated that hierarchical goal-setting enabled people to make more progress towards goals that appeared less actionable. This suggests that supporting hierarchical goal-setting with chatbots is a promising approach to helping people who don’t know how to pursue their goals.

[BibTex]

González Cruz, H., Prentice, M., Lieder, F. ’What Do You Want in Life and How Can You Get There?’ An Evaluation of a Hierarchical Goal-Setting Chatbot 13th Annual meeting of the Society for the Science of Motivation, Abstract of presentation at the 13th SSM Virtual Congress, Society for the Science of Motivation, Virtual Congress, May 2021 (conference)

[BibTex]

Measuring and modelling how people learn how to plan and how people adapt their planning strategies the to structure of the environment
Measuring and modelling how people learn how to plan and how people adapt their planning strategies the to structure of the environment

He, R., Jain, Y. R., Lieder, F.

International Conference on Cognitive Modeling, International Conference on Cognitive Modeling, 2021 (conference)

Abstract
Often we find ourselves in unknown situations where we have to make a decision based on reasoning upon experiences. However, it is still unclear how people choose which pieces of information to take into account to achieve well-informed decisions. Answering this question requires an understanding of human metacognitive learning, that is how do people learn how to think. In this study, we focus on a special kind of metacognitive learning, namely how people learn how to plan and how their mechanisms of metacognitive learning adapt the planning strategies to the structures of the environment. We first measured people's adaptation to different environments via a process-tracing paradigm that externalises planning. Then we introduced and fitted novel metacognitive reinforcement learning algorithms to model the underlying learning mechanisms, which enabled us insights into the learning behaviour. Model-based analysis suggested two sources of maladaptation: no learning and reluctance to explore new alternatives.

link (url) Project Page [BibTex]

2020

A Gamified App that Helps People Overcome Self-Limiting Beliefs by Promoting Metacognition
A Gamified App that Helps People Overcome Self-Limiting Beliefs by Promoting Metacognition

Amo, V., Lieder, F.

SIG 8 Meets SIG 16, SIG 8 Meets SIG 16, September 2020 (conference) Accepted

Abstract
Previous research has shown that approaching learning with a growth mindset is key for maintaining motivation and overcoming setbacks. Mindsets are systems of beliefs that people hold to be true. They influence a person's attitudes, thoughts, and emotions when they learn something new or encounter challenges. In clinical psychology, metareasoning (reflecting on one's mental processes) and meta-awareness (recognizing thoughts as mental events instead of equating them to reality) have proven effective for overcoming maladaptive thinking styles. Hence, they are potentially an effective method for overcoming self-limiting beliefs in other domains as well. However, the potential of integrating assisted metacognition into mindset interventions has not been explored yet. Here, we propose that guiding and training people on how to leverage metareasoning and meta-awareness for overcoming self-limiting beliefs can significantly enhance the effectiveness of mindset interventions. To test this hypothesis, we develop a gamified mobile application that guides and trains people to use metacognitive strategies based on Cognitive Restructuring (CR) and Acceptance Commitment Therapy (ACT) techniques. The application helps users to identify and overcome self-limiting beliefs by working with aversive emotions when they are triggered by fixed mindsets in real-life situations. Our app aims to help people sustain their motivation to learn when they face inner obstacles (e.g. anxiety, frustration, and demotivation). We expect the application to be an effective tool for helping people better understand and develop the metacognitive skills of emotion regulation and self-regulation that are needed to overcome self-limiting beliefs and develop growth mindsets.

A gamified app that helps people overcome self-limiting beliefs by promoting metacognition Project Page [BibTex]

no image
How to navigate everyday distractions: Leveraging optimal feedback to train attention control

Wirzberger, M., Lado, A., Eckerstorfer, L., Oreshnikov, I., Passy, J., Stock, A., Shenhav, A., Lieder, F.

Proceedings of the 42nd Annual Meeting of the Cognitive Science Society, Cognitive Science Society, July 2020 (conference)

Abstract
To stay focused on their chosen tasks, people have to inhibit distractions. The underlying attention control skills can improve through reinforcement learning, which can be accelerated by giving feedback. We applied the theory of metacognitive reinforcement learning to develop a training app that gives people optimal feedback on their attention control while they are working or studying. In an eight-day field experiment with 99 participants, we investigated the effect of this training on people's productivity, sustained attention, and self-control. Compared to a control condition without feedback, we found that participants receiving optimal feedback learned to focus increasingly better (f = .08, p < .01) and achieved higher productivity scores (f = .19, p < .01) during the training. In addition, they evaluated their productivity more accurately (r = .12, p < .01). However, due to asymmetric attrition problems, these findings need to be taken with a grain of salt.

How to navigate everyday distractions: Leveraging optimal feedback to train attention control DOI Project Page [BibTex]

no image
Measuring the Costs of Planning

Felso, V., Jain, Y. R., Lieder, F.

Proceedings of the 42nd Annual Meeting of the Cognitive Science Society, (Editors: S. Denison and M. Mack and Y. Zu and B. C. Armstrong), Cognitive Science Society, CogSci 2020, July 2020 (conference) Accepted

Abstract
Which information is worth considering depends on how much effort it would take to acquire and process it. From this perspective people’s tendency to neglect considering the long-term consequences of their actions (present bias) might reflect that looking further into the future becomes increasingly more effortful. In this work, we introduce and validate the use of Bayesian Inverse Reinforcement Learning (BIRL) for measuring individual differences in the subjective costs of planning. We extend the resource-rational model of human planning introduced by Callaway, Lieder, et al. (2018) by parameterizing the cost of planning. Using BIRL, we show that increased subjective cost for considering future outcomes may be associated with both the present bias and acting without planning. Our results highlight testing the causal effects of the cost of planning on both present bias and mental effort avoidance as a promising direction for future work.

Project Page Project Page [BibTex]

Felso, V., Jain, Y. R., Lieder, F. Measuring the Costs of Planning Proceedings of the 42nd Annual Meeting of the Cognitive Science Society, (Editors: S. Denison and M. Mack and Y. Zu and B. C. Armstrong), Cognitive Science Society, CogSci 2020, July 2020 (conference) Accepted

Project Page Project Page [BibTex]

no image
Leveraging Machine Learning to Automatically Derive Robust Planning Strategies from Biased Models of the Environment

Kemtur, A., Jain, Y. R., Mehta, A., Callaway, F., Consul, S., Stojcheski, J., Lieder, F.

Proceedings of the 42nd Annual Meeting of the Cognitive Science Society, Cognitive Science Society, CogSci 2020, July 2020, Anirudha Kemtur and Yash Raj Jain contributed equally to this publication. (conference)

Abstract
Teaching clever heuristics is a promising approach to improve decision-making. We can leverage machine learning to discover clever strategies automatically. Current methods require an accurate model of the decision problems people face in real life. But most models are misspecified because of limited information and cognitive biases. To address this problem we develop strategy discovery methods that are robust to model misspecification. Robustness is achieved by model-ing model-misspecification and handling uncertainty about the real-world according to Bayesian inference. We translate our methods into an intelligent tutor that automatically discovers and teaches robust planning strategies. Our robust cognitive tutor significantly improved human decision-making when the model was so biased that conventional cognitive tutors were no longer effective. These findings highlight that our robust strategy discovery methods are a significant step towards leveraging artificial intelligence to improve human decision-making in the real world.

Leveraging Machine Learning to Automatically Derive Robust Planning Strategies from Biased Models of the Environment Project Page Project Page [BibTex]

Kemtur, A., Jain, Y. R., Mehta, A., Callaway, F., Consul, S., Stojcheski, J., Lieder, F. Leveraging Machine Learning to Automatically Derive Robust Planning Strategies from Biased Models of the Environment Proceedings of the 42nd Annual Meeting of the Cognitive Science Society, Cognitive Science Society, CogSci 2020, July 2020, Anirudha Kemtur and Yash Raj Jain contributed equally to this publication. (conference)

Leveraging Machine Learning to Automatically Derive Robust Planning Strategies from Biased Models of the Environment Project Page Project Page [BibTex]

no image
ACTrain: Ein KI-basiertes Aufmerksamkeitstraining für die Wissensarbeit

Wirzberger, M., Oreshnikov, I., Passy, J., Lado, A., Shenhav, A., Lieder, F.

66th Spring Conference of the German Ergonomics Society, 2020 (conference)

Abstract
Unser digitales Zeitalter lebt von Informationen und stellt unsere begrenzte Verarbeitungskapazität damit täglich auf die Probe. Gerade in der Wissensarbeit haben ständige Ablenkungen erhebliche Leistungseinbußen zur Folge. Unsere intelligente Anwendung ACTrain setzt genau an dieser Stelle an und verwandelt Computertätigkeiten in eine Trainingshalle für den Geist. Feedback auf Basis maschineller Lernverfahren zeigt anschaulich den Wert auf, sich nicht von einer selbst gewählten Aufgabe ablenken zu lassen. Diese metakognitive Einsicht soll zum Durchhalten motivieren und das zugrunde liegende Fertigkeitsniveau der Aufmerksamkeitskontrolle stärken. In laufenden Feldexperimenten untersuchen wir die Frage, ob das Training mit diesem optimalen Feedback die Aufmerksamkeits- und Selbstkontrollfertigkeiten im Vergleich zu einer Kontrollgruppe ohne Feedback verbessern kann.

link (url) Project Page [BibTex]

2019

How do people learn how to plan?
How do people learn how to plan?

Jain, Y. R., Gupta, S., Rakesh, V., Dayan, P., Callaway, F., Lieder, F.

2019 Conference on Cognitive Computational Neuroscience, September 2019 (conference)

Abstract
How does the brain learn how to plan? We reverse-engineer people's underlying learning mechanisms by combining rational process models of cognitive plasticity with recently developed empirical methods that allow us to trace the temporal evolution of people's planning strategies. We find that our Learned Value of Computation model (LVOC) accurately captures people's average learning curve. However, there were also substantial individual differences in metacognitive learning that are best understood in terms of multiple different learning mechanisms-including strategy selection learning. Furthermore, we observed that LVOC could not fully capture people's ability to adaptively decide when to stop planning. We successfully extended the LVOC model to address these discrepancies. Our models broadly capture people's ability to improve their decision mechanisms and represent a significant step towards reverse-engineering how the brain learns increasingly effective cognitive strategies through its interaction with the environment.

How do people learn to plan? How do people learn to plan? Project Page [BibTex]

2019

Jain, Y. R., Gupta, S., Rakesh, V., Dayan, P., Callaway, F., Lieder, F. How do people learn how to plan? 2019 Conference on Cognitive Computational Neuroscience, September 2019 (conference)

How do people learn to plan? How do people learn to plan? Project Page [BibTex]

no image
Testing Computational Models of Goal Pursuit

Mohnert, F., Tosic, M., Lieder, F.

2019 Conference on Cognitive Computational Neuroscience,, CCN2019, September 2019 (conference)

Abstract
Goals are essential to human cognition and behavior. But how do we pursue them? To address this question, we model how capacity limits on planning and attention shape the computational mechanisms of human goal pursuit. We test the predictions of a simple model based on previous theories in a behavioral experiment. The results show that to fully capture how people pursue their goals it is critical to account for people’s limited attention in addition to their limited planning. Our findings elucidate the cognitive constraints that shape human goal pursuit and point to an improved model of human goal pursuit that can reliably predict which goals a person will achieve and which goals they will struggle to pursue effectively.

link (url) DOI Project Page [BibTex]

Mohnert, F., Tosic, M., Lieder, F. Testing Computational Models of Goal Pursuit 2019 Conference on Cognitive Computational Neuroscience,, CCN2019, September 2019 (conference)

link (url) DOI Project Page [BibTex]

no image
Measuring How People Learn How to Plan

Jain, Y. R., Callaway, F., Lieder, F.

In Proceedings 41st Annual Meeting of the Cognitive Science Society, pages: 1956-1962, CogSci2019, 41st Annual Meeting of the Cognitive Science Society, July 2019 (inproceedings)

Abstract
The human mind has an unparalleled ability to acquire complex cognitive skills, discover new strategies, and refine its ways of thinking and decision-making; these phenomena are collectively known as cognitive plasticity. One important manifestation of cognitive plasticity is learning to make better–more far-sighted–decisions via planning. A serious obstacle to studying how people learn how to plan is that cognitive plasticity is even more difficult to observe than cognitive strategies are. To address this problem, we develop a computational microscope for measuring cognitive plasticity and validate it on simulated and empirical data. Our approach employs a process tracing paradigm recording signatures of human planning and how they change over time. We then invert a generative model of the recorded changes to infer the underlying cognitive plasticity. Our computational microscope measures cognitive plasticity significantly more accurately than simpler approaches, and it correctly detected the effect of an external manipulation known to promote cognitive plasticity. We illustrate how computational microscopes can be used to gain new insights into the time course of metacognitive learning and to test theories of cognitive development and hypotheses about the nature of cognitive plasticity. Future work will leverage our computational microscope to reverse-engineer the learning mechanisms enabling people to acquire complex cognitive skills such as planning and problem solving.

link (url) Project Page Project Page [BibTex]

Jain, Y. R., Callaway, F., Lieder, F. Measuring How People Learn How to Plan In Proceedings 41st Annual Meeting of the Cognitive Science Society, pages: 1956-1962, CogSci2019, 41st Annual Meeting of the Cognitive Science Society, July 2019 (inproceedings)

link (url) Project Page Project Page [BibTex]

How should we incentivize learning? An optimal feedback mechanism for educational games and online courses
How should we incentivize learning? An optimal feedback mechanism for educational games and online courses

Xu, L., Wirzberger, M., Lieder, F.

41st Annual Meeting of the Cognitive Science Society, July 2019 (conference)

Abstract
Online courses offer much-needed opportunities for lifelong self-directed learning, but people rarely follow through on their noble intentions to complete them. To increase student retention educational software often uses game elements to motivate students to engage in and persist in learning activities. However, gamification only works when it is done properly, and there is currently no principled method that educational software could use to achieve this. We develop a principled feedback mechanism for encouraging good study choices and persistence in self-directed learning environments. Rather than giving performance feedback, our method rewards the learner's efforts with optimal brain points that convey the value of practice. To derive these optimal brain points, we applied the theory of optimal gamification to a mathematical model of skill acquisition. In contrast to hand-designed incentive structures, optimal brain points are constructed in such a way that the incentive system cannot be gamed. Evaluating our method in a behavioral experiment, we find that optimal brain points significantly increased the proportion of participants who instead of exploiting an inefficient skill they already knew-attempted to learn a difficult but more efficient skill, persisted through failure, and succeeded to master the new skill. Our method provides a principled approach to designing incentive structures and feedback mechanisms for educational games and online courses. We are optimistic that optimal brain points will prove useful for increasing student retention and helping people overcome the motivational obstacles that stand in the way of self-directed lifelong learning.

link (url) Project Page [BibTex]

no image
Extending Rationality

Pothos, E. M., Busemeyer, J. R., Pleskac, T., Yearsley, J. M., Tenenbaum, J. B., Goodman, N. D., Tessler, M. H., Griffiths, T. L., Lieder, F., Hertwig, R., Pachur, T., Leuker, C., Shiffrin, R. M.

Proceedings of the 41st Annual Conference of the Cognitive Science Society, pages: 39-40, CogSci 2019, July 2019 (conference)

Proceedings of the 41st Annual Conference of the Cognitive Science Society [BibTex]

Pothos, E. M., Busemeyer, J. R., Pleskac, T., Yearsley, J. M., Tenenbaum, J. B., Goodman, N. D., Tessler, M. H., Griffiths, T. L., Lieder, F., Hertwig, R., Pachur, T., Leuker, C., Shiffrin, R. M. Extending Rationality Proceedings of the 41st Annual Conference of the Cognitive Science Society, pages: 39-40, CogSci 2019, July 2019 (conference)

Proceedings of the 41st Annual Conference of the Cognitive Science Society [BibTex]

no image
What’s in the Adaptive Toolbox and How Do People Choose From It? Rational Models of Strategy Selection in Risky Choice

Mohnert, F., Pachur, T., Lieder, F.

41st Annual Meeting of the Cognitive Science Society, July 2019 (conference)

Abstract
Although process data indicates that people often rely on various (often heuristic) strategies to choose between risky options, our models of heuristics cannot predict people's choices very accurately. To address this challenge, it has been proposed that people adaptively choose from a toolbox of simple strategies. But which strategies are contained in this toolbox? And how do people decide when to use which decision strategy? Here, we develop a model according to which each person selects decisions strategies rationally from their personal toolbox; our model allows one to infer which strategies are contained in the cognitive toolbox of an individual decision-maker and specifies when she will use which strategy. Using cross-validation on an empirical data set, we find that this rational model of strategy selection from a personal adaptive toolbox predicts people's choices better than any single strategy (even when it is allowed to vary across participants) and better than previously proposed toolbox models. Our model comparisons show that both inferring the toolbox and rational strategy selection are critical for accurately predicting people's risky choices. Furthermore, our model-based data analysis reveals considerable individual differences in the set of strategies people are equipped with and how they choose among them; these individual differences could partly explain why some people make better choices than others. These findings represent an important step towards a complete formalization of the notion that people select their cognitive strategies from a personal adaptive toolbox.

link (url) Project Page Project Page [BibTex]

no image
Measuring How People Learn How to Plan

Jain, Y. R., Callaway, F., Lieder, F.

pages: 357-361, RLDM 2019, July 2019 (conference)

Abstract
The human mind has an unparalleled ability to acquire complex cognitive skills, discover new strategies, and refine its ways of thinking and decision-making; these phenomena are collectively known as cognitive plasticity. One important manifestation of cognitive plasticity is learning to make better – more far-sighted – decisions via planning. A serious obstacle to studying how people learn how to plan is that cognitive plasticity is even more difficult to observe than cognitive strategies are. To address this problem, we develop a computational microscope for measuring cognitive plasticity and validate it on simulated and empirical data. Our approach employs a process tracing paradigm recording signatures of human planning and how they change over time. We then invert a generative model of the recorded changes to infer the underlying cognitive plasticity. Our computational microscope measures cognitive plasticity significantly more accurately than simpler approaches, and it correctly detected the effect of an external manipulation known to promote cognitive plasticity. We illustrate how computational microscopes can be used to gain new insights into the time course of metacognitive learning and to test theories of cognitive development and hypotheses about the nature of cognitive plasticity. Future work will leverage our computational microscope to reverse-engineer the learning mechanisms enabling people to acquire complex cognitive skills such as planning and problem solving.

link (url) [BibTex]

Jain, Y. R., Callaway, F., Lieder, F. Measuring How People Learn How to Plan pages: 357-361, RLDM 2019, July 2019 (conference)

link (url) [BibTex]

no image
A Cognitive Tutor for Helping People Overcome Present Bias

Lieder, F., Callaway, F., Jain, Y. R., Krueger, P. M., Das, P., Gul, S., Griffiths, T. L.

RLDM 2019, July 2019, Falk Lieder and Frederick Callaway contributed equally to this publication. (conference)

Abstract
People's reliance on suboptimal heuristics gives rise to a plethora of cognitive biases in decision-making including the present bias, which denotes people's tendency to be overly swayed by an action's immediate costs/benefits rather than its more important long-term consequences. One approach to helping people overcome such biases is to teach them better decision strategies. But which strategies should we teach them? And how can we teach them effectively? Here, we leverage an automatic method for discovering rational heuristics and insights into how people acquire cognitive skills to develop an intelligent tutor that teaches people how to make better decisions. As a proof of concept, we derive the optimal planning strategy for a simple model of situations where people fall prey to the present bias. Our cognitive tutor teaches people this optimal planning strategy by giving them metacognitive feedback on how they plan in a 3-step sequential decision-making task. Our tutor's feedback is designed to maximally accelerate people's metacognitive reinforcement learning towards the optimal planning strategy. A series of four experiments confirmed that training with the cognitive tutor significantly reduced present bias and improved people's decision-making competency: Experiment 1 demonstrated that the cognitive tutor's feedback can help participants discover far-sighted planning strategies. Experiment 2 found that this training effect transfers to more complex environments. Experiment 3 found that these transfer effects are retained for at least 24 hours after the training. Finally, Experiment 4 found that practicing with the cognitive tutor can have additional benefits over being told the strategy in words. The results suggest that promoting metacognitive reinforcement learning with optimal feedback is a promising approach to improving the human mind.

DOI Project Page [BibTex]

Lieder, F., Callaway, F., Jain, Y. R., Krueger, P. M., Das, P., Gul, S., Griffiths, T. L. A Cognitive Tutor for Helping People Overcome Present Bias RLDM 2019, July 2019, Falk Lieder and Frederick Callaway contributed equally to this publication. (conference)

DOI Project Page [BibTex]

no image
Introducing the Decision Advisor: A simple online tool that helps people overcome cognitive biases and experience less regret in real-life decisions

lawama, G., Greenberg, S., Moore, D., Lieder, F.

40th Annual Meeting of the Society for Judgement and Decision Making, June 2019 (conference)

Abstract
Cognitive biases shape many decisions people come to regret. To help people overcome these biases, Clear-erThinking.org developed a free online tool, called the Decision Advisor (https://programs.clearerthinking.org/decisionmaker.html). The Decision Advisor assists people in big real-life decisions by prompting them to generate more alternatives, guiding them to evaluate their alternatives according to principles of decision analysis, and educates them about pertinent biases while they are making their decision. In a within-subjects experiment, 99 participants reported significantly fewer biases and less regret for a decision supported by the Decision Advisor than for a previous unassisted decision.

DOI [BibTex]

lawama, G., Greenberg, S., Moore, D., Lieder, F. Introducing the Decision Advisor: A simple online tool that helps people overcome cognitive biases and experience less regret in real-life decisions 40th Annual Meeting of the Society for Judgement and Decision Making, June 2019 (conference)

DOI [BibTex]

no image
The Goal Characteristics (GC) questionannaire: A comprehensive measure for goals’ content, attainability, interestingness, and usefulness

Iwama, G., Wirzberger, M., Lieder, F.

40th Annual Meeting of the Society for Judgement and Decision Making, June 2019 (conference)

Abstract
Many studies have investigated how goal characteristics affect goal achievement. However, most of them considered only a small number of characteristics and the psychometric properties of their measures remains unclear. To overcome these limitations, we developed and validated a comprehensive questionnaire of goal characteristics with four subscales - measuring the goal’s content, attainability, interestingness, and usefulness respectively. 590 participants completed the questionnaire online. A confirmatory factor analysis supported the four subscales and their structure. The GC questionnaire (https://osf.io/qfhup) can be easily applied to investigate goal setting, pursuit and adjustment in a wide range of contexts.

DOI Project Page Project Page [BibTex]

no image
Remediating Cognitive Decline with Cognitive Tutors

Das, P., Callaway, F., Griffiths, T. L., Lieder, F.

RLDM 2019, 2019 (conference)

Abstract
As people age, their cognitive abilities tend to deteriorate, including their ability to make complex plans. To remediate this cognitive decline, many commercial brain training programs target basic cognitive capacities, such as working memory. We have recently developed an alternative approach: intelligent tutors that teach people cognitive strategies for making the best possible use of their limited cognitive resources. Here, we apply this approach to improve older adults' planning skills. In a process-tracing experiment we found that the decline in planning performance may be partly because older adults use less effective planning strategies. We also found that, with practice, both older and younger adults learned more effective planning strategies from experience. But despite these gains there was still room for improvement-especially for older people. In a second experiment, we let older and younger adults train their planning skills with an intelligent cognitive tutor that teaches optimal planning strategies via metacognitive feedback. We found that practicing planning with this intelligent tutor allowed older adults to catch up to their younger counterparts. These findings suggest that intelligent tutors that teach clever cognitive strategies can help aging decision-makers stay sharp.

DOI Project Page [BibTex]

Das, P., Callaway, F., Griffiths, T. L., Lieder, F. Remediating Cognitive Decline with Cognitive Tutors RLDM 2019, 2019 (conference)

DOI Project Page [BibTex]

2018

no image
Discovering and Teaching Optimal Planning Strategies

Lieder, F., Callaway, F., Krueger, P. M., Das, P., Griffiths, T. L., Gul, S.

In The 14th biannual conference of the German Society for Cognitive Science, GK, September 2018, Falk Lieder and Frederick Callaway contributed equally to this publication. (inproceedings)

Abstract
How should we think and decide, and how can we learn to make better decisions? To address these questions we formalize the discovery of cognitive strategies as a metacognitive reinforcement learning problem. This formulation leads to a computational method for deriving optimal cognitive strategies and a feedback mechanism for accelerating the process by which people learn how to make better decisions. As a proof of concept, we apply our approach to develop an intelligent system that teaches people optimal planning stratgies. Our training program combines a novel process-tracing paradigm that makes peoples latent planning strategies observable with an intelligent system that gives people feedback on how their planning strategy could be improved. The pedagogy of our intelligent tutor is based on the theory that people discover their cognitive strategies through metacognitive reinforcement learning. Concretely, the tutor’s feedback is designed to maximally accelerate people’s metacognitive reinforcement learning towards the optimal cognitive strategy. A series of four experiments confirmed that training with the cognitive tutor significantly improved people’s decision-making competency: Experiment 1 demonstrated that the cognitive tutor’s feedback accelerates participants’ metacognitive learning. Experiment 2 found that this training effect transfers to more difficult planning problems in more complex environments. Experiment 3 found that these transfer effects are retained for at least 24 hours after the training. Finally, Experiment 4 found that practicing with the cognitive tutor conveys additional benefits above and beyond verbal description of the optimal planning strategy. The results suggest that promoting metacognitive reinforcement learning with optimal feedback is a promising approach to improving the human mind.

link (url) Project Page [BibTex]

2018

Lieder, F., Callaway, F., Krueger, P. M., Das, P., Griffiths, T. L., Gul, S. Discovering and Teaching Optimal Planning Strategies In The 14th biannual conference of the German Society for Cognitive Science, GK, September 2018, Falk Lieder and Frederick Callaway contributed equally to this publication. (inproceedings)

link (url) Project Page [BibTex]

no image
Discovering Rational Heuristics for Risky Choice

Gul, S., Krueger, P. M., Callaway, F., Griffiths, T. L., Lieder, F.

The 14th biannual conference of the German Society for Cognitive Science, GK, The 14th biannual conference of the German Society for Cognitive Science, GK, September 2018 (conference)

Abstract
How should we think and decide to make the best possible use of our precious time and limited cognitive resources? And how do people’s cognitive strategies compare to this ideal? We study these questions in the domain of multi-alternative risky choice using the methodology of resource-rational analysis. To answer the first question, we leverage a new meta-level reinforcement learning algorithm to derive optimal heuristics for four different risky choice environments. We find that our method rediscovers two fast-and-frugal heuristics that people are known to use, namely Take-The-Best and choosing randomly, as resource-rational strategies for specific environments. Our method also discovered a novel heuristic that combines elements of Take-The-Best and Satisficing. To answer the second question, we use the Mouselab paradigm to measure how people’s decision strategies compare to the predictions of our resource-rational analysis. We found that our resource-rational analysis correctly predicted which strategies people use and under which conditions they use them. While people generally tend to make rational use of their limited resources overall, their strategy choices do not always fully exploit the structure of each decision problem. Overall, people’s decision operations were about 88% as resource-rational as they could possibly be. A formal model comparison confirmed that our resource-rational model explained people’s decision strategies significantly better than the Directed Cognition model of Gabaix et al. (2006). Our study is a proof-of-concept that optimal cognitive strategies can be automatically derived from the principle of resource-rationality. Our results suggest that resource-rational analysis is a promising approach for uncovering people’s cognitive strategies and revisiting the debate about human rationality with a more realistic normative standard.

link (url) Project Page [BibTex]

Gul, S., Krueger, P. M., Callaway, F., Griffiths, T. L., Lieder, F. Discovering Rational Heuristics for Risky Choice The 14th biannual conference of the German Society for Cognitive Science, GK, The 14th biannual conference of the German Society for Cognitive Science, GK, September 2018 (conference)

link (url) Project Page [BibTex]

no image
Learning to Select Computations

Callaway, F., Gul, S., Krueger, P. M., Griffiths, T. L., Lieder, F.

In Uncertainty in Artificial Intelligence: Proceedings of the Thirty-Fourth Conference, August 2018, Frederick Callaway and Sayan Gul and Falk Lieder contributed equally to this publication. (inproceedings)

Abstract
The efficient use of limited computational resources is an essential ingredient of intelligence. Selecting computations optimally according to rational metareasoning would achieve this, but this is computationally intractable. Inspired by psychology and neuroscience, we propose the first concrete and domain-general learning algorithm for approximating the optimal selection of computations: Bayesian metalevel policy search (BMPS). We derive this general, sample-efficient search algorithm for a computation-selecting metalevel policy based on the insight that the value of information lies between the myopic value of information and the value of perfect information. We evaluate BMPS on three increasingly difficult metareasoning problems: when to terminate computation, how to allocate computation between competing options, and planning. Across all three domains, BMPS achieved near-optimal performance and compared favorably to previously proposed metareasoning heuristics. Finally, we demonstrate the practical utility of BMPS in an emergency management scenario, even accounting for the overhead of metareasoning.

link (url) Project Page [BibTex]

Callaway, F., Gul, S., Krueger, P. M., Griffiths, T. L., Lieder, F. Learning to Select Computations In Uncertainty in Artificial Intelligence: Proceedings of the Thirty-Fourth Conference, August 2018, Frederick Callaway and Sayan Gul and Falk Lieder contributed equally to this publication. (inproceedings)

link (url) Project Page [BibTex]

A resource-rational analysis of human planning
A resource-rational analysis of human planning

Callaway, F., Lieder, F., Das, P., Gul, S., Krueger, P. M., Griffiths, T. L.

In Proceedings of the 40th Annual Conference of the Cognitive Science Society, May 2018, Frederick Callaway and Falk Lieder contributed equally to this publication. (inproceedings)

Abstract
People's cognitive strategies are jointly shaped by function and computational constraints. Resource-rational analysis leverages these constraints to derive rational models of people's cognitive strategies from the assumption that people make rational use of limited cognitive resources. We present a resource-rational analysis of planning and evaluate its predictions in a newly developed process tracing paradigm. In Experiment 1, we find that a resource-rational planning strategy predicts the process by which people plan more accurately than previous models of planning. Furthermore, in Experiment 2, we find that it also captures how people's planning strategies adapt to the structure of the environment. In addition, our approach allows us to quantify for the first time how close people's planning strategies are to being resource-rational and to characterize in which ways they conform to and deviate from optimal planning.

DOI [BibTex]

Callaway, F., Lieder, F., Das, P., Gul, S., Krueger, P. M., Griffiths, T. L. A resource-rational analysis of human planning In Proceedings of the 40th Annual Conference of the Cognitive Science Society, May 2018, Frederick Callaway and Falk Lieder contributed equally to this publication. (inproceedings)

DOI [BibTex]

2017

no image
Optimal gamification can help people procrastinate less

Lieder, F., Griffiths, T. L.

Annual Meeting of the Society for Judgment and Decision Making, Annual Meeting of the Society for Judgment and Decision Making, November 2017 (conference)

[BibTex]

2017

Lieder, F., Griffiths, T. L. Optimal gamification can help people procrastinate less Annual Meeting of the Society for Judgment and Decision Making, Annual Meeting of the Society for Judgment and Decision Making, November 2017 (conference)

[BibTex]

no image
An automatic method for discovering rational heuristics for risky choice

Lieder, F., Krueger, P. M., Griffiths, T. L.

In Proceedings of the 39th Annual Meeting of the Cognitive Science Society. Austin, TX: Cognitive Science Society, 2017, Falk Lieder and Paul M. Krueger contributed equally to this publication. (inproceedings)

Abstract
What is the optimal way to make a decision given that your time is limited and your cognitive resources are bounded? To answer this question, we formalized the bounded optimal decision process as the solution to a meta-level Markov decision process whose actions are costly computations. We approximated the optimal solution and evaluated its predictions against human choice behavior in the Mouselab paradigm, which is widely used to study decision strategies. Our computational method rediscovered well-known heuristic strategies and the conditions under which they are used, as well as novel heuristics. A Mouselab experiment confirmed our model’s main predictions. These findings are a proof-of-concept that optimal cognitive strategies can be automatically derived as the rational use of finite time and bounded cognitive resources.

Project Page [BibTex]

Lieder, F., Krueger, P. M., Griffiths, T. L. An automatic method for discovering rational heuristics for risky choice In Proceedings of the 39th Annual Meeting of the Cognitive Science Society. Austin, TX: Cognitive Science Society, 2017, Falk Lieder and Paul M. Krueger contributed equally to this publication. (inproceedings)

Project Page [BibTex]

no image
A reward shaping method for promoting metacognitive learning

Lieder, F., Krueger, P. M., Callaway, F., Griffiths, T. L.

In Proceedings of the Third Multidisciplinary Conference on Reinforcement Learning and Decision-Making, 2017 (inproceedings)

Project Page [BibTex]

Lieder, F., Krueger, P. M., Callaway, F., Griffiths, T. L. A reward shaping method for promoting metacognitive learning In Proceedings of the Third Multidisciplinary Conference on Reinforcement Learning and Decision-Making, 2017 (inproceedings)

Project Page [BibTex]

no image
When does bounded-optimal metareasoning favor few cognitive systems?

Milli, S., Lieder, F., Griffiths, T. L.

In AAAI Conference on Artificial Intelligence, 31, 2017 (inproceedings)

[BibTex]

Milli, S., Lieder, F., Griffiths, T. L. When does bounded-optimal metareasoning favor few cognitive systems? In AAAI Conference on Artificial Intelligence, 31, 2017 (inproceedings)

[BibTex]

no image
The Structure of Goal Systems Predicts Human Performance

Bourgin, D., Lieder, F., Reichman, D., Talmon, N., Griffiths, T.

In Proceedings of the 39th Annual Meeting of the Cognitive Science Society, 2017 (inproceedings)

[BibTex]

Bourgin, D., Lieder, F., Reichman, D., Talmon, N., Griffiths, T. The Structure of Goal Systems Predicts Human Performance In Proceedings of the 39th Annual Meeting of the Cognitive Science Society, 2017 (inproceedings)

[BibTex]

no image
Learning to (mis) allocate control: maltransfer can lead to self-control failure

Bustamante, L., Lieder, F., Musslick, S., Shenhav, A., Cohen, J.

In The 3rd Multidisciplinary Conference on Reinforcement Learning and Decision Making. Ann Arbor, Michigan, 2017 (inproceedings)

[BibTex]

Bustamante, L., Lieder, F., Musslick, S., Shenhav, A., Cohen, J. Learning to (mis) allocate control: maltransfer can lead to self-control failure In The 3rd Multidisciplinary Conference on Reinforcement Learning and Decision Making. Ann Arbor, Michigan, 2017 (inproceedings)

[BibTex]

no image
Mouselab-MDP: A new paradigm for tracing how people plan

Callaway, F., Lieder, F., Krueger, P. M., Griffiths, T. L.

In The 3rd multidisciplinary conference on reinforcement learning and decision making, 2017 (inproceedings)

Project Page [BibTex]

Callaway, F., Lieder, F., Krueger, P. M., Griffiths, T. L. Mouselab-MDP: A new paradigm for tracing how people plan In The 3rd multidisciplinary conference on reinforcement learning and decision making, 2017 (inproceedings)

Project Page [BibTex]

no image
Enhancing metacognitive reinforcement learning using reward structures and feedback

Krueger, P. M., Lieder, F., Griffiths, T. L.

In Proceedings of the 39th Annual Meeting of the Cognitive Science Society, 2017 (inproceedings)

Project Page [BibTex]

Krueger, P. M., Lieder, F., Griffiths, T. L. Enhancing metacognitive reinforcement learning using reward structures and feedback In Proceedings of the 39th Annual Meeting of the Cognitive Science Society, 2017 (inproceedings)

Project Page [BibTex]

no image
Helping people choose subgoals with sparse pseudo rewards

Callaway, F., Lieder, F., Griffiths, T. L.

In Proceedings of the Third Multidisciplinary Conference on Reinforcement Learning and Decision Making, 2017 (inproceedings)

[BibTex]

Callaway, F., Lieder, F., Griffiths, T. L. Helping people choose subgoals with sparse pseudo rewards In Proceedings of the Third Multidisciplinary Conference on Reinforcement Learning and Decision Making, 2017 (inproceedings)

[BibTex]

2016

no image
Helping people make better decisions using optimal gamification

Lieder, F., Griffiths, T. L.

In Proceedings of the 38th Annual Conference of the Cognitive Science Society, 2016 (inproceedings)

Abstract
Game elements like points and levels are a popular tool to nudge and engage students and customers. Yet, no theory can tell us which incentive structures work and how to design them. Here we connect the practice of gamification to the theory of reward shaping in reinforcement learning. We leverage this connection to develop a method for designing effective incentive structures and delineating when gamification will succeed from when it will fail. We evaluate our method in two behavioral experiments. The results of the first experiment demonstrate that incentive structures designed by our method help people make better, less short-sighted decisions and avoid the pitfalls of less principled approaches. The results of the second experiment illustrate that such incentive structures can be effectively implemented using game elements like points and badges. These results suggest that our method provides a principled way to leverage gamification to help people make better decisions.

link (url) Project Page [BibTex]

2016

Lieder, F., Griffiths, T. L. Helping people make better decisions using optimal gamification In Proceedings of the 38th Annual Conference of the Cognitive Science Society, 2016 (inproceedings)

link (url) Project Page [BibTex]

2015

no image
When to use which heuristic: A rational solution to the strategy selection problem

Lieder, F., Griffiths, T. L.

In Proceedings of the 37th Annual Conference of the Cognitive Science Society, 2015 (inproceedings)

Abstract
The human mind appears to be equipped with a toolbox full of cognitive strategies, but how do people decide when to use which strategy? We leverage rational metareasoning to derive a rational solution to this problem and apply it to decision making under uncertainty. The resulting theory reconciles the two poles of the debate about human rationality by proposing that people gradually learn to make rational use of fallible heuristics. We evaluate this theory against empirical data and existing accounts of strategy selection (i.e. SSL and RELACS). Our results suggest that while SSL and RELACS can explain people's ability to adapt to homogeneous environments in which all decision problems are of the same type, rational metareasoning can additionally explain people's ability to adapt to heterogeneous environments and flexibly switch strategies from one decision to the next.

link (url) Project Page [BibTex]

2015

Lieder, F., Griffiths, T. L. When to use which heuristic: A rational solution to the strategy selection problem In Proceedings of the 37th Annual Conference of the Cognitive Science Society, 2015 (inproceedings)

link (url) Project Page [BibTex]

no image
Children and Adults Differ in their Strategies for Social Learning

Lieder, F., Sim, Z. L., Hu, J. C., Griffiths, T. L., Xu, F.

In Proceedings of the 37th Annual Conference of the Cognitive Science Society, 2015 (inproceedings)

Abstract
Adults and children rely heavily on other people’s testimony. However, domains of knowledge where there is no consensus on the truth are likely to result in conflicting testimonies. Previous research has demonstrated that in these cases, learners look towards the majority opinion to make decisions. However, it remains unclear how learners evaluate social information, given that considering either the overall valence, or the number of testimonies, or both may lead to different conclusions. We therefore formalized several social learning strategies and compared them to the performance of adults and children. We find that children use different strategies than adults. This suggests that the development of social learning may involve the acquisition of cognitive strategies.

link (url) [BibTex]

Lieder, F., Sim, Z. L., Hu, J. C., Griffiths, T. L., Xu, F. Children and Adults Differ in their Strategies for Social Learning In Proceedings of the 37th Annual Conference of the Cognitive Science Society, 2015 (inproceedings)

link (url) [BibTex]

no image
Learning from others: Adult and child strategies in assessing conflicting ratings

Hu, J., Lieder, F., Griffiths, T. L., Xu, F.

In Biennial Meeting of the Society for Research in Child Development, Philadelphia, Pennsylvania, USA, 2015 (inproceedings)

[BibTex]

Hu, J., Lieder, F., Griffiths, T. L., Xu, F. Learning from others: Adult and child strategies in assessing conflicting ratings In Biennial Meeting of the Society for Research in Child Development, Philadelphia, Pennsylvania, USA, 2015 (inproceedings)

[BibTex]

no image
Utility-weighted sampling in decisions from experience

Lieder, F., Griffiths, T. L., Hsu, M.

In The 2nd Multidisciplinary Conference on Reinforcement Learning and Decision Making, 2015 (inproceedings)

[BibTex]

Lieder, F., Griffiths, T. L., Hsu, M. Utility-weighted sampling in decisions from experience In The 2nd Multidisciplinary Conference on Reinforcement Learning and Decision Making, 2015 (inproceedings)

[BibTex]