ORCID Profile
0000-0003-2318-6576
Current Organisation
Netherlands Comprehensive Cancer Organisation (IKNL)
Does something not look right? The information on this page has been harvested from data sources that may not be up to date. We continue to work with information providers to improve coverage and quality. To report an issue, use the Feedback Form.
Related Links
Publications
Psychological Science needs a standard practice of reporting the reliability of cognitive behavioural measurements
Publisher: Center for Open Science
Date: 05-04-2018
Abstract: Psychological science relies on behavioural measures to assess cognitive processing however, the field has not yet developed a tradition of routinely examining the reliability of these behavioural measures. Reliable measures are essential to draw robust inferences from statistical analyses, while subpar reliability has severe implications for the measures’ validity and interpretation. Without examining and reporting the reliability of cognitive behavioural measurements, it is near impossible to ascertain whether results are robust or have arisen largely from measurement error. In this paper we propose that researchers adopt a standard practice of estimating and reporting the reliability of behavioural assessments. We illustrate this proposal using an ex le from experimental psychopathology, the dot-probe task although we argue that reporting reliability is relevant across fields (e.g. social cognition and cognitive psychology). We explore several implications of low measurement reliability, and the detrimental impact that failure to assess measurement reliability has on interpretability and comparison of results and therefore research quality. We argue that the field needs to a) report measurement reliability as routine practice so that we can b) develop more reliable assessment tools. To provide some guidance on estimating and reporting reliability, we describe bootstrapped split half estimation and IntraClass Correlation Coefficient procedures to estimate internal consistency and test-retest reliability, respectively. For future researchers to build upon current results it is imperative that all researchers provide sufficient psychometric information to estimate the accuracy of inferences and inform further development of cognitive behavioural assessments.
Capturing Dynamics of Biased Attention: Are New Attention Variability Measures the Way Forward?
Publisher: Public Library of Science (PLoS)
Date: 22-11-2016
Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements
Publisher: SAGE Publications
Date: 06-11-2019
Abstract: Psychological science relies on behavioral measures to assess cognitive processing however, the field has not yet developed a tradition of routinely examining the reliability of these behavioral measures. Reliable measures are essential to draw robust inferences from statistical analyses, and subpar reliability has severe implications for measures’ validity and interpretation. Without examining and reporting the reliability of measurements used in an analysis, it is nearly impossible to ascertain whether results are robust or have arisen largely from measurement error. In this article, we propose that researchers adopt a standard practice of estimating and reporting the reliability of behavioral assessments of cognitive processing. We illustrate the need for this practice using an ex le from experimental psychopathology, the dot-probe task, although we argue that reporting reliability is relevant across fields (e.g., social cognition and cognitive psychology). We explore several implications of low measurement reliability and the detrimental impact that failure to assess measurement reliability has on interpretability and comparison of results and therefore research quality. We argue that researchers in the field of cognition need to report measurement reliability as routine practice so that more reliable assessment tools can be developed. To provide some guidance on estimating and reporting reliability, we describe the use of bootstrapped split-half estimation and intraclass correlation coefficients to estimate internal consistency and test-retest reliability, respectively. For future researchers to build upon current results, it is imperative that all researchers provide psychometric information sufficient for estimating the accuracy of inferences and informing further development of cognitive-behavioral assessments.
A meta-analysis of bias at baseline in RCTs of attention bias modification: No evidence for dot-probe bias towards threat in clinical anxiety and PTSD.
Publisher: American Psychological Association (APA)
Date: 08-2019
DOI: 10.1037/ABN0000406
Abstract: Considerable effort and funding have been spent on developing Attention Bias Modification (ABM) as a treatment for anxiety disorders, theorized to exert therapeutic effects through reduction of a tendency to orient attention toward threat. However, meta-analytical evidence that clinical anxiety is characterized by threat-related attention bias is thin. The largest meta-analysis to date included dot-probe data for n = 337 clinically anxious in iduals. Baseline measures of biased attention obtained in ABM RCTs form an additional body of data that has not previously been meta-analyzed. This article presents a meta-analysis of threat-related dot-probe bias measured at baseline for 1,005 clinically anxious in iduals enrolled in 13 ABM RCTs. Random-effects meta-analysis indicated no evidence that the mean bias index (BI) differed from zero (k = 13, n = 1005, mean BI = 1.8 ms, SE = 1.26 ms, p = .144, 95% confidence interval [-0.6, 4.3]. Additional Bayes factor analyses also supported the point-zero hypothesis (BF10 = .23), whereas interval-based analysis indicated that mean bias in clinical anxiety is unlikely to extend beyond the 0 to 5 ms interval. Findings are discussed with respect to strengths (relatively large s les, possible bypassing of publication bias), limitations (lack of control comparison, repurposing data, specificity to dot-probe data), and theoretical and practical context. We suggest that it should no longer be assumed that clinically anxious in iduals are characterized by selective attention toward threat. Clinically anxious in iduals enrolled in RCTs for Attention Bias Modification are not characterized by threat-related attention bias at baseline. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
A meta-analysis of bias at baseline in RCTs of attention bias modification: no evidence for dot-probe bias towards threat in clinical anxiety and PTSD.
Publisher: Center for Open Science
Date: 06-07-2018
Abstract: Background: Considerable effort and funding have been spent on developing Attention Bias Modification (ABM) as a treatment for anxiety disorders, theorized to exert therapeutic effects through reduction of a tendency to orient attention towards threat. However, meta-analytical evidence that clinical anxiety is characterized by threat-related attention bias is thin. The largest meta-analysis to date included dot-probe data for n=337 clinically anxious in iduals. Baseline measures of biased attention obtained in ABM RCTs form an additional body of data that has not previously been meta-analyzed. Method: This paper presents a meta-analysis of threat-related dot-probe bias measured at baseline for 1005 clinically anxious in iduals enrolled in 13 ABM RCTs.Results: Random-effects meta-analysis indicated no evidence that the mean bias index (BI) differed from zero (k= 13, n= 1005, mean BI = 1.8 ms, SE = 1.26 ms, p = .144, 95% CI [-0.6 - 4.3]. Additional Bayes factor analyses also supported the point-zero hypothesis (BF10 = .23), whereas interval-based analysis indicated that mean bias in clinical anxiety is unlikely to extend beyond the 0 to 5 ms interval. Discussion: Findings are discussed with respect to strengths (relatively large s les, possible bypassing of publication bias), limitations (lack of control comparison, repurposing data, specificity to dot-probe data), and theoretical and practical context. We suggest that it should no longer be assumed that clinically anxious in iduals are characterized by selective attention towards threat. Conclusion: Clinically anxious in iduals enrolled in RCTs for Attention Bias Modification are not characterized by threat-related attention bias at baseline.
Related Organisations
Netherlands Comprehensive Cancer Organisation (IKNL)
Location: Netherlands
University Of Oxford
Location: United Kingdom of Great Britain and Northern Ireland
Related Funding Activities
No related grants have been discovered for Anne-Wil Kruijt.