David F. Urschler - Do we really know what we are talking about? The prevalence of well- and ill-defined psychological constructs in 150 meta-analytic reviews. - Perspectives on Scientific Error 2024
See osf.io/ayfek/
The importance of scientific rigor has been in the spotlight of psychological research in the past decade. For example, previous research has provided comprehensive guidelines to increase replicability, and highlighted the importance of theory for improving psychological science. However, we argue that the crucial aspect of construct clarity has not yet been sufficiently considered, because concept clarification of psychological constructs is a prerequisite for reliable and valid scientific endeavors. Our reasoning is underpinned by previous research that has revealed that several eminent psychological constructs have been ill-defined. For example, a review revealed that empathy has been defined in 43 different ways, which has had a negative impact on both research and practice. Moreover, constructs of interest have been solely defined by their operationalizations (i.e., operational definitions) that undermines conceptual clarity. Consequently, we examined the prevalence of well- and ill-defined constructs in psychological science.
Given that meta-analytic reviews on a certain topic are more likely to be cited and to influence the field than a single research paper, we sampled 150 meta-analyses from Psychological Bulletin across the period from 1990 to 2017. We focused on Psychological Bulletin because Psychological Bulletin is a renowned outlet for meta-analytic reviews across all psychological disciplines. To answer whether the construct(s) of interest were defined, and if yes, how were they defined, each meta-analytic review was coded by two independent coders. Additionally, we coded several descriptive characteristics of each meta-analytic review (e.g., number of included constructs, discipline, first- and last-authors gender, construct’s valence; the full-list of coded characteristics will be available at OSF).
Our results revealed that the 150 meta-analytic reviews contained 359 constructs in total. Out of these 359 constructs, 140 (39%) constructs were defined. Out of the 140 defined constructs, for seven a conceptual definition, for 50 a homeostatic property cluster definition (a construct is defined by a cluster of features that regularly but not exceptionlessly co-occur), and for 83 an operational definition was provided. The main findings are, that the majority of constructs in meta-analytic reviews were not defined, and if they were defined, the majority of the constructs were lacking a conceptional definition. In our discussion, we argue for an increased emphasis on conceptual definitions, which, in turn, further improves scientific rigor in psychological research in the future.
Негізгі бет Ғылым және технология David F. Urschler - Do we really know what we are talking about? - PoSE 2024
Пікірлер