This task assess two types of reinforcement learning (RL): model-free and model-based RL. In this task, participants make two sequential decisions that navigate them through two "stages" defined by different stimuli. First-stage choices are associated with one of two second stages (e.g., 2a and 2b): one first-stage choice leads to 2a 70% of the time and 2b 30% of the time, while the opposite is true of the other first-stage choice (i.e. 2a occurs 30% of the time and 2b occurs 70% of the time). Each second-stage choice is associated with some probability of receiving a reward. This probability changes slowly over time, requiring continuous learning in order to succeed at the task. Because the goal of the subject is to maximize rewards in the second stage, ideal performance would entail identifying the most rewarding second stage (e.g., 2a) and making first-stage choices that make this result more likely (e.g., the first-stage choice that results in 2a 70% of the time).
In the decision-making literature, two categories of reinforcement learning (RL) have gained prominence: model-free and model-based RL. Model-free RL proceeds by updating and caching values (rewards) associated with actions in particular states. It is a computationally efficient procedure at the time of decision (decisions only require retrieving the cached state-action value), but it is insensitive to rapid transitions in environment structure or reward contingencies. In contrast, model-based RL computes the value of actions by explicitly using a model of the environment and calculating the value of different action trajectories at the time of the decision. These two descriptions represent extremes along a spectrum, and human decision-making has been described as a combination of the two. Distinguishing between specific types of RL, specifically understanding model-based learning, may shed light on processes underlying self-regulation and behavior.
[+] PMCID, PUBMED ID, or CITATION
Text Citation: Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215.
This measure has not been measured yet.
This measure has not been influenced yet.
This measure has not been validated yet.
The Science of Behavior Change (SOBC) program seeks to promote basic research on the initiation, personalization and maintenance of behavior change. By integrating work across disciplines, this effort will lead to an improved understanding of the underlying principles of behavior change. The SOBC program aims to implement a mechanisms-focused, experimental medicine approach to behavior change research and to develop the tools required to implement such an approach. The experimental medicine approach involves: identifying an intervention target, developing measures to permit verification of the target, engaging the target through experimentation or intervention, and testing the degree to which target engagement produces the desired behavior change.
Within the SOBC Measures Repository, researchers have access to measures of mechanistic targets that have been (or are in the processing of being) validated by SOBC Research Network Members and other experts in the field. The SOBC Validation Process includes three important stages of evaluation for each proposed measure: Identification, Measurement, and Influence.
The first stage of validation requires a measure to be Identified within the field; there must be theoretical support for the specific measure of the proposed mechanistic target or potential mechanism of behavior change. This evidence may include references for the proposed measure, or theoretical support for the construct that the proposed measure is intended to assess. The second stage of validation requires demonstration that the level and change in level of the chosen mechanistic target can be Measured with the proposed measure (assay). For example, if the proposed measure is a questionnaire, the score on the measure should indicate the activity of the target process, and it must have strong psychometric properties. The third stage of validation requires demonstration that the measure can be Influenced; there must be evidence that the measured target is malleable and responsive to manipulation. Evidence relating to each stage includes at least one peer-reviewed publication or original data presentation (if no peer-reviewed research is available to support the claim) and is evaluated by SOBC Research Network Members and experts in the field.
Once a measure has gone through these three stages, it will then either be Validated or Not validated according to SOBC Research Network standards. If a measure is Validated, then change in the measured target was reliably associated with Behavior Change. If a measure is Not validated, then change in the measured target was not reliably associated with Behavior Change. Why would we share measures that are not validated? The SOBC Research Network values open, rigorous, and transparent research. Our goal is to make meaningful progress and develop replicable and effective interventions in behavior change science. Therefore, the SOBC sees value in providing other researchers in the field with information regarding measures that work and measures that fall short for specific targets. Further, a measure that is not validated for one target in one population may be validated in another target or population.
Want to learn more? For any questions regarding the SOBC Validation Process or Measures Repository, please email firstname.lastname@example.org.
Has the mechanism been identified as a potential target for behavior change? This section summarizes theoretical support for the mechanism.
Have the psychometric properties of this measure been assessed? This section includes information such as content validity, internal consistency, and test-retest reliability.
Has a study manipulation led to change in the mechanism? This section addresses evidence that this measure is modifiable by experimental manipulation or clinical intervention.
Has a change in this mechanism been associated with behavior change? This section addresses empirical evidence that causing change in the measure reliably produces subsequent behavior change.