Our research approach consisted of four steps: (1) selection of outcome and causal conditions; (2) selection of relevant case studies; (3) data collection and coding of case study characteristics; and (4) comparative analysis using fsQCA.

Selection of outcome and causal conditions

We first drew on relevant literature to understand how past studies have measured outcomes and causal conditions in planned relocations. While there is no consensus about which conceptual framework is most applicable for assessing hazard-related relocation outcomes, a range of approaches were identified in the literature from diverse disciplinary perspectives (Supplementary Section 3). Each conceptual framework has advantages and disadvantages, as most were developed for other purposes. Some frameworks are better suited to assess outcomes at the level of a whole community, such as livability, whereas others are more tailored to individual scales, such as well-being. Some frameworks miss key dimensions specific to hazard-related relocation outcomes, such as cultural aspects, governance or exposure reduction. Many of these frameworks were developed to understand outcomes in analogous development resettlement, forced migration or disaster recovery contexts and have limited applicability for climate or hazard-related planned relocation cases. Here, we selected the SLA²⁷ to understand how access to various assets changes as a result of relocation²². SLA is well suited as an outcome measure because it: (1) can be used at the community scale, (2) enables comparison across cases with different baselines and (3) allows for multidimensional consideration of human needs (natural, social, financial, human and physical, with the addition of cultural dimensions to make SLA+^19,28).

For causal conditions, we reviewed literature to identify diverse planning, hazard, community stakeholder and governance factors that past research suggests impact relocation outcomes (Supplementary Section 4). In considering these factors, we narrowed our focus to a subset that met three criteria: (1) suggested to be important for sustainable livelihood outcomes specifically, (2) consistently measurable across these 14 cases and (3) viable areas for possible planning intervention. Table 1 in Supplementary Section 4 provides details regarding factor selection.

Case selection

Until recently, a global database of hazard-related planned relocation cases did not exist, constituting an obstacle to comparative research. In this analysis, we draw from a new database of planned relocation cases initiated after 1970 published by the Platform on Disaster Displacement and the Kaldor Center in 2021¹⁴. While we were revising this manuscript, a mapping of success typologies in managed retreat programmes was published⁹, providing another compilation of cases for future comparative analyses.

From the global database, we selected planned relocation cases that met certain selection criteria. Case selection involves trade-offs between more cases for increased sample and fewer cases to ensure consistency across cases and meaningful comparisons. Our case criteria were: (1) the physical relocation has already occurred (‘completed’ in the database), (2) relocation from one site of origin to one destination (‘type A’ in this database¹⁴), (3) one of the identified natural hazards was floods (including in the context of riverine settings, coastal erosion and tsunamis) and (4) adequate documentation. When this search was conducted in January 2021, the database included 73 cases identified from the initial mapping of English language literature that met these first three conditions. (Note that after a June 2021 review of Spanish, French and Portuguese literature, the database now has 14 additional cases meeting criteria (1) to (3) for a total of 87 cases.) However, most (81%) of these original 73 cases did not meet the criterion of adequate documentation, which were as follows: (1) at least one piece of evidence for nearly all six SLA+ suboutcome conditions and five relocation planning decision factors and (2) at least one article using a consistent data collection method of interviews with key stakeholders. We recognize that the adequate documentation criterion may have biased selection towards cases in the developed and English-speaking world and cases that were more well-known and thus more well-resourced, which constitutes a limitation to this research. We ultimately selected the 14 cases of planned relocation that met these criteria, varying in completion year between 1981 and 2016.

Data collection and coding of causal and outcome conditions per case

To gain in-depth knowledge of all 14 selected planned relocations, the coding team (E.R.B. and A.B.) conducted case study literature reviews through: document identification, data extraction by theme, content analysis and calibration and narrative synthesis.

Document identification

In January–March 2021, we identified diverse documents—including academic articles, white papers, government planning materials and media articles—through systematic searches on Google Scholar, Google and Google News. We used these search engines, rather than Scopus or Web of Science because we wanted to capture evidence and develop case knowledge about relocation outcomes and planning factors from documents produced by government actors (for example, US Army Corps of Engineers), the media (for example, radio shows and local newspapers) and community members (for example, blogs and YouTube videos) in addition to academic researchers (for example, peer-reviewed papers and unpublished theses). To identify relevant and diverse documents, we used search terms ‘Village/Community of origin, Country’ and ‘Relocation’ or ‘Resettlement’ or ‘Retreat’ and considered the top 25 results in each search engine. We scanned each document and selected only those where the discussion of the relocation case was substantive; documents containing one to two sentences about the case were not included. All documents referenced for case descriptions and calibration decisions are listed in Table 1 of Supplementary Section 1.

Data extraction by theme

Using the qualitative data analysis tool NVivo (QSR International), the coding team then systematically extracted all relevant data per relocation case, which were categorized into themes pertaining to background context, planning decision factors and outcomes. For background, we considered evidence about the hazard(s) involved, the process duration (years of relocation decision initiation and completion of the physical move), the locations origin and destination site and additional geographic and socioeconomic context. For relocation planning decisions, we considered evidence about community engagement, distance, scale, speed and transfer dynamics (see Table 1 for definitions). For outcomes, we considered evidence about relocated people’s change in access to assets required for SLA+, including physical, economic, natural, human, social and cultural categories (see Table 1 for definitions, aligned with the SLA+ assets adapted from previous studies^19,27,28). To ensure robustness, we used NVivo’s percentage agreement calculator and found an average of 84% agreement for the extraction of relevant outcome and causal conditions across the two members of the coder team.

For some cases, information about all outcome and causal conditions was not available in the reviewed documents. If we were unsure about a specific calibration decision or if less information was available, we contacted the authors of reviewed documents for further information and verification. We triangulated with Google Earth to verify the distances between origin and destination sites and used these estimates if there was a discrepancy. While some cases have experienced multiple relocations over time, we considered only the distance between the most recent origin and destination sites.

Content analysis and calibration

Next, we undertook a content analysis of the available data for each relocation decision factor and outcome category. For each of the six outcome categories, we made evaluative judgments of negative or positive valence of the evidence following the approach of refs. ^19,28. For example, we coded increased access to electricity through the installation of solar panels in the destination site as a positive physical outcome, while we coded a decrease in access to health centres as a negative human outcome. On the basis of theoretically informed and predetermined calibration criteria (Table 1 of Supplementary Section 2), we aggregated available evidence and assigned each case a score per outcome category and relocation planning decision factor. Cases had differing quantities of evidence for each category, which we accounted for through a six-point scoring system based on percentages of available evidence (discussed below). To ensure robustness, two members of the research team undertook the coding procedure for each case independently. Then, both team members collectively reviewed each case, allowing for a consensus interpretation through ‘negotiated agreement’ to establish intercoder reliability⁵⁷.

Narrative synthesis

After coding all the outcome and relocation planning conditions, we then summarized our coding into narratives for each of the 14 cases. Supplementary Section 1 contains information on all cases, including a short description, replication evidence for calibration of causal and outcome conditions and a list of references for each case, while Supplementary Section 2 summarizes the calibration approaches used for causal and outcome conditions and descriptive statistics across cases.

Assessing causality through fsQCA

To determine what pathways lead to more positive or negative livelihood outcomes, we compared patterns across 14 cases using fsQCA. This involved multiple steps: calibration, assess necessity and sufficiency, robustness tests and case verification. All analyses were conducted using fs/QCA V.3.0 software⁵⁸.

The fsQCA is a comparative-case analytic method developed by refs. ^38,59 on the basis of principles of Boolean algebra and fuzzy-set theory. It examines set memberships of cases (determined from qualitative and quantitative data) to identify if conditions are necessary or sufficient for explaining outcomes. While there are many relevant approaches in the universe of analytical techniques, including principal component analysis and cluster analysis^39,60, among others, fsQCA had several critical advantages for this analysis. First, it “both bridge[s] and transcend[s] the qualitative–quantitative divide”⁶¹: as for case study approaches, the method retains in-depth case complexity but like large-sample statistical approaches, it enables some degree of generalizability through robust comparisons. Second, it is ideal for medium sample sizes, which are usually defined as being between 10 and 50 cases³⁸. Third, it allows for complex causality where multiple conditions act in combination to influence an outcome. Fourth, this approach identifies multiple pathways to the same outcome that coexist (‘equifinality’), addressing concerns about local context dependence. Finally, in contrast to the earlier qualitative comparative analysis variant that requires ‘crisp’ condition and outcome scores (binary 0 or 1), the more recent fsQCA approach allows for nuanced scoring along a ‘fuzzy’ scale (range from 0 to 1). The fsQCA applications are increasing rapidly⁶², including recent papers examining post-typhoon relocations¹⁸ and shelter projects⁴⁹ in the Philippines, post-tsunami recovery in India⁶³ and post-hurricane recovery in New Orleans⁶⁴. The fsQCA has also been used^41,42,65 for meta-analysis when questions are about configurations of case study literature (as in this study) rather than effect size⁶⁶. Qualitative comparative analysis is a useful method when quantitative meta-analysis falls short, such as when interventions are complex and when there is heterogeneity between cases that cannot be explained through statistical methods. In such circumstances, fsQCA can “replace standard fall back on narrative synthesis and usefully suggest ways in which a combination of characteristics are associated with improved outcomes” (p.13)⁴⁰.

Calibration

To apply this method, we first undertook a process known as calibration: we converted the raw case data into set membership scores ranging from 0 to 1 using predefined criteria for both outcomes and relocation planning factors (Table 1 of Supplementary Section 2). For livelihood outcomes, each case was assigned a score for all six asset categories along a fuzzy-set six-point scale. As explained in Table 1 of Supplementary Section 2, we considered all the available evidence for each suboutcome category and assigned scores of 0, 0.2, 0.4, 0.6, 0.8 or 1 on the basis of per cent of available evidence indicating positive or negative outcomes. For relocation planning factors with quantitative data (scale, speed and distance), we used the standard approach to direct calibration using the fs/QCA 3.0 software function ‘calibrate’ which uses log-odds⁵⁹. The remaining two factors were calibrated indirectly using six-point scales based on case knowledge. Community engagement measured the frequency and inclusivity of involvement of relocating community members at three stages of the process: initiation, site selection and site development. Transfer dynamics measured the level and quality of synchronicity of transfer dynamics at three stages of the process: decision, during move and destination. Table 1 of Supplementary Section 2 provides further details on calibration approaches.

Assess necessity and sufficiency

We then assessed the necessity and sufficiency of conditions in explaining outcomes. In fsQCA, necessary and sufficient relationships can be defined in terms of set relations. Necessary relationships occur when a condition is observed in (nearly) all cases with the outcome; this implies that the set of cases with the outcome is a subset of the cases with the condition. Sufficient relationships occur when the outcome is observed if the condition, or combination of conditions, is present; in other words, these cases are a subset of cases with the outcome. Both necessity and sufficiency analyses use two measures commonly used to assess a QCA: consistency and coverage. Consistency measures the strength of the relationship between condition and outcome or the degree to which one set is a perfect subset of another. Values range from 0 to 1 and, while there are no universally defined standards, a value of 0.9 is the generally accepted cutoff point for reliable analysis of necessity⁶⁷ and 0.8 for sufficiency⁶⁸. Coverage, by contrast, measures how well a subset condition explains an outcome, again with values ranging from 0 to 1. A highly consistent condition with low coverage has low empirical importance. Consistency is related to predictability and somewhat analogous to a correlation coefficient, whereas coverage is related to relevance and somewhat analogous to an R² value, although the QCA community cautions against such comparisons.

Following ref. ³⁸, we first undertook an analysis of the necessity of each individual condition (Table 3 and Tables 3 and 4 of Supplementary Section 2). We tested for both the presence and absence of each condition in explaining cases with more positive livelihood outcomes and, as a robustness check, also tested for more negative livelihood outcomes. Since no conditions were necessary for either positive or negative outcomes, we followed standard practice and next analysed the sufficiency of combinations of conditions.

To analyse sufficiency, we used fs/QCA software to generate a ‘truth table’ of all possible combinations of causal conditions (Table 5 of Supplementary Section 2). Each row represents a possible combination of conditions; since our analysis has five conditions, there are 2⁵ or 32 possible rows. Each case is assigned to its corresponding row, while some rows are not represented by real cases and are called ‘logical remainders’. The higher the number of conditions included in an analysis, the higher the number of rows and therefore also logical remainders. Analyses with many conditions and few cases face challenges with limited diversity and validity of results, making the ideal ratio four to five conditions for 12–16 cases, as in our analysis⁶⁹. Following a procedure known as minimization, which is the logical simplification of set relations among conditions and the outcome, the truth table rows are assessed for sufficient combinations of conditions.

The fsQCA software generates three types of solutions (complex, parsimonious and intermediate) representing different treatments of remainders as counterfactuals; see Table 6 of Supplementary Section 2. We follow commonly accepted practice⁶⁸ and focus on the intermediate solution based on theoretically informed assumptions about the connections between the presence or absence of conditions and the outcome. In this analysis, the intermediate solution assumes that the presence of community engagement and absence of distance are associated with better outcomes and gives preference to these factors when there were tied prime implicants but no assumptions are made for speed, scale or transfer dynamics given the absence of consensus in the theoretical literature.

Robustness tests

Next, we conducted a series of robustness tests, including changes in consistency thresholds, calibration approaches and removal of cases^41,70. Given that this analysis includes 14 cases, we only considered a frequency threshold of one case per row⁶⁸. We also undertook additional robustness tests with differing approaches to calibration of the outcome condition. In the primary analysis, we used the unweighted mean to provide a holistic measure of outcomes across all six SLA+ categories. However, recognizing that this averaging approach may lessen the influence of extremes and has the tendency to bring cases to the 0.5 threshold score, we also considered alternative weighting approaches through robustness tests. We considered: (1) all minimum outcomes (cases calibrated to the lowest score of the six SLA+ categories), (2) all maximum outcomes (each case calibrated to the highest score of the six SLA+ categories) and (3) a combination of minimum and maximum outcomes (cases with on average negative outcomes calibrated to the lowest score, while cases with on average positive outcomes calibrated to the highest score). Table 7 of Supplementary Section 2 includes results of all robustness tests.

Case verification

Finally, we examined each pathway to ascertain if it challenges or refines existing insights from individual cases and broader literature. We qualitatively compared across case studies to better understand these pathways and identify future research directions.

Limitations

Our analysis is limited by the availability of data in documents summarizing these relocation cases. Our coding is designed to capture information about outcomes and planning decisions in these documents, which may be distorted by biases of the authors and entities that published the studies and by pernicious or misleading socioeconomic forces such as colonialism.

There are also challenges arising from the heterogeneity of sources, including the range of years of case completion and document publication. Each publication reflects circumstances at the time when each was written, which vary by document and may not reflect the status of the case at present. Thus, the passage of time is a compounding factor, as the assessment of outcomes may vary widely depending on whether the assessment in each document took place one year or one decade after the completion of the physical move. Supplementary Section 1 includes a table with dates of each publication considered per case to capture this variation. To ameliorate biases, future research efforts should monitor and evaluate relocation outcomes longitudinally over time.

Additionally, the measurement of each construct has limitations and results should be interpreted accordingly. For example, we measured speed as the duration of time between initiation and the physical move of most people but this fails to capture how the construction of physical infrastructure and services may have lagged behind or proceeded the movement of people. Similarly, we measured scale as number of households but other proxies for this construct could be the spatial size of the lots and homes. Additionally, the way factors such as speed and distance are perceived varies across cultural contexts—what is considered ‘slow’ in Australia may be considered ‘fast’ in Vanuatu, for example. Future research considering alternative measurements for each construct is needed.

Further, our analysis is limited by the focus on the community scale; relocating communities are not homogenous and both relocation planning decisions and outcomes may vary for people of different ages, genders, abilities, relationship to land, status as renters or owners and status as long-term residents or newcomers, among other axes of diversity^11,28,71,72. Further research is needed to better understand relocation outcomes for individuals with differing intersectional identities.

Finally, we acknowledge that there are limitations to the types of insights generated from the analysis of a medium sample of 14 cases, from relying on search engines for document identification, from using fsQCA for a configurational meta-analysis and from our scoring approaches. Additional research methods, including both larger sample statistical meta-analyses and in-depth single case studies, are needed to verify and extend these findings about pathways towards more sustainable livelihood outcomes in planned relocations.