Using causal loop diagrams to develop evaluative research propositions: Opportunities and challenges in applications to Nature Based Solutions
System Dynamics Review 2023
Alvarado M, Garrett J, Fullam J, Lovell R, Guell C, Taylor T, Garside R, Zandersen M, Wheeler BW
DOI : 10.1002/sdr.1756
URL : https://www.doi.org/10.1002/sdr.1756
Abstract
Causal loop diagrams (CLDs) are often used to provide an overview of important systemic elements related to an issue, rather than to inform empirical evaluations (studies which assess changes following an intervention using observed data). We suggest that empirical evaluations may benefit from the development of systems-informed research propositions (specific testable causal assumptions with an emphasis on feedback loops) used to guide subsequent data collection, hypothesis testing and interpretation. We describe a qualitative systems-thinking informed approach building on preexisting CLDs, published evidence, and expert/stakeholder consultation and reflect on our experience applying this to the early stages of two nature-based solution (NBS) evaluations. We reflect on our experience and suggest that CLDs can be usefully employed to develop systems-informed research propositions to inform subsequent empirical evaluation. This may lead to novel policy-relevant research propositions which differ substantially from effectiveness-oriented (“did it work?”) research questions.
Lay Summary
Causal loop diagrams (CLDs) are commonly used for systems analysis, offering insights into complex issues. However, their application in empirical evaluations is limited. This paper proposes a novel approach: integrating systems thinking to develop research propositions with a focus on feedback loops for empirical testing. The study illustrates this method using two Nature-Based Solution (NBS) evaluations, emphasizing the potential to uncover policy-relevant research propositions distinct from traditional effectiveness-oriented questions.
In the face of major challenges like climate change and non-communicable diseases, large-scale interventions, such as policy changes or infrastructure projects, are crucial. However, evaluating their impact is challenging due to factors beyond researchers' control. Traditional evaluation methods, like randomized control trials, often fall short in these cases. This study introduces a fresh perspective, suggesting the use of causal loop diagrams (CLDs) to formulate systems-informed research propositions. These propositions focus on understanding the dynamic relationships and feedback loops within complex systems, providing a more holistic basis for subsequent empirical evaluations.
The proposed approach involves five steps: representing the underlying system, understanding the intervention, identifying links between the intervention and the system, generating systems-informed research propositions, and finally, sense-checking these propositions with stakeholders. The paper illustrates this process through two case studies: street trees in urban areas and wetland restoration in rural communities.
By integrating systems thinking into the early stages of evaluation, the study suggests that researchers can develop more nuanced research propositions which go beyond traditional effectiveness questions and consider the dynamic interplay of elements within the system. This approach holds promise for enhancing the understanding of how interventions interact with complex systems, ultimately contributing to more informed policy decisions.