Material for the tutorial paper dynConfiR: An R package for sequential sampling models of decision confidence

This repository contains the data and code used in the paper: dynConfiR: An R package for sequential sampling models of decision confidence (Hellmann, S., Zehetleitner, M., & Rausch, M., 2024, Preprint on PsyArXiv (https://osf.io/preprints/psyarxiv/e354s)).

Structure:

• The folder application_Law contains data and code for the Example section of the paper:

  • the data by Law & Lee (Ng et al., 2021) (together with a reamde for the data)
  • the file Script_fitSeqSampConfModels_Law.R, which contains the code for model fitting and prediction,
  • the files IC_analysis1.R and IC_analysis2.R, which contain code for analysis and visualisation of the quantitative model comparison
  • the folder plotscripts contains some R-scripts to produce the figures in the manuscript
  • the folders autosave_all_standard_models and autosave_fixed_noiseparameters contain logging files and saved fitting results for individual participants and models (with subfolders for each fitted model)
  • parfits.RData, parfits_MTLNR.RData, and parfits_fixed_and_free.RData contain the fitted parameters for the standard models and the standard models together with the restricted models with fixed noise parameters, respectively
  • fitsandpredictions.RData and fitsandpredictions_allfits.RData contain the fitted parameters as well as predicted distributions for the standard models (including MTLNR) and the standard models together with the restricted models with fixed noise parameters, respectively

• The folders density_precision_dynaViTE, density_precision_RM, and density_precision_MTLNR contain the code for the Precision analysis section in the manuscript, each containing:

  • a R-script Script_density_precision... containing the main code for the computations and the visualization of the results,
  • helper scripts paramerer_prior.R and helper_compute_probs.R (each containing identical code for both folders), used to simulate parameter sets and to compute the densities for different precisions
  • several .RData-files containing the intermediate and final results of the analysis

• The folder parameter_recovery (and parameter_recovery_MTLNR for this model specifically) with code used for the Parameter recovery analysis section of the manuscript. To reproduce the figures in the manuscript, the script in parameter_recovery_MTLNR should be used as it loads the results from the other folder). The folders each contains:

  • The script Script_par_rec.R (Script_par_rec_MTLNR.R), which contains the main recovery analysis, from simulating artificial data and fitting the models to the simulated data, it sources several functions from
  • the folder helper_fct.R, which contains R-files each defining functions:
    • to collect and combine previous parameter fits from empirical studies (read_and_collect_previous_fits.R, the result is saved in collected_fits_models.RData),
    • to sample random proportions of confidence samples from a Dirichlet distribution (fun_sample_rating_props.R, the information for this is saved in collected_rating_proportions_and_alpha.RData)
    • to simulate artificial data using the sampled parameter sets and confidence proportions (helper_simulate_artificialdata_fivesteps.R)
    • and fitting a specific model to given data and combine with known true parameters (fitting_function_parrec_fivesteps.R)
  • the folder prevfits including .RData-files that contain estimated parameters from model fitting to different empirical data sets (different files) for different models (contained in the same file)
  • the folder saved_details, containing all_par_samples_list.RData (with all parameter sets used as true parameters for the recovery) and the folder fit_results containing subfolders for each model, each containing the fitting results for each iteration of the recovery simulation (another subfolder simulated_data is not included but will be generated when running the main script for the parameter recovery)
  • the file saved_results.RData contains all collected results from the recovery analysis
  • the script gather_plot_results.R (gather_plot_results_MTLNR.R), which visualizes the results of the parameter recovery study

• The folder model_recovery with code used for the Model recovery analysis section of the manuscript. It contains:

  • The script Script_model_rec.R and Script_extend_model_rec_MTLNR.R, which contain the main recovery analysis, from simulating artificial data and fitting the models to the simulated data. The second script includes an extension for the MTLNR model, which was added at a later stage. Both scripts source several functions from
  • the folder helper_fct.R, which contains R-files each defining functions (pretty much the same as for parameter recovery; only the fitting function deviates slightly):
    • to collect and combine previous parameter fits from empirical studies (read_and_collect_previous_fits.R, the result is saved in collected_fits_models.RData),
    • to sample random proportions of confidence samples from a Dirichlet distribution (fun_sample_rating_props.R, the information for this is saved in collected_rating_proportions_and_alpha.RData)
    • to simulate artificial data using the sampled parameter sets and confidence proportions (helper_simulate_artificialdata_fivesteps.R)
    • and fitting a specific model to given data (fitting_function_parrec_fivesteps.R)
  • the folder prevfits including .RData-files that contain estimated parameters from model fitting to different empirical data sets (different files) for different models (contained in the same file)
  • the folder saved_details, containing all_par_samples_list.RData and all_par_samples_list_MTLNR.RData (with all parameter sets used as true parameters for the recovery) and the folder fit_results containing subfolders for each generative and fitted model combination, each containing the fitting results for each iteration of the recovery simulation (another subfolder simulated_data is not included but will be generated when running the main script for the parameter recovery)
    • the file list_w_all_recoveries.RData in the subfolder fit_results contains all collected results from the model recovery for each simulation
  • the files group_BMS_results.RData and group_BMS_results_AIC.RData contain the results from the group-level comparison in the model recovery

• dynConfiR-source package file (.tar.gz). This the 1.1.1-version of the package which was used for the publication, with likelihood and fitting functions, used for the computations. A more recent version may be available on GitHub and on CRAN. The newer versions on GitHub and CRAN may produce different results. The source package may be installed via

  install.packages("dynWEV_1.1.1.tar.gz", type = "source", dependencies=TRUE,repos="http://a.cran.mirror") 

  The more recent versions may be installed using install.packages("dynConfiR")

Dependencies and compatibility:

The analyses use different other R packages. The dependencies for the dynConfiR-package should be handled, when installing the package. The scripts use the package rstudioapi, which should be installed with any RStudio version, to find the script path for setting the working directory. The package parallel is used for parallelization but should also be installed in any R distribution.

The following additional packages are used:

install.packages(c("BayesFactor", "tidyverse", "ggh4x", "ggpubr", "scales"))

References

Ng, L. C. H., Law, F. H. F., Lam, A. M. W., Or, C. C.-F., & Lee, A. L. F. (2021). Metacognitive adaptation revealed in serial dependence of visual confidence judgments. Journal of Vision, 21 (9), 2487. https://doi.org/10.1167/jov.21.9.2487

Contact

For comments, remarks, and questions please contact me: sebastian.hellmann@tum.de