#ManyDesignsCarbon

Open Science · Crowd Science · Metascience

Fifty-five Crowd-Sourced Designs
Assessing Carbon Pricing Support

Armando Holzknecht^* · Rene Schwaiger^* · Esther Blanco · Jürgen Huber · Michael Kirchler
5 project coordinators · 108 co-authors · 55 research teams · University of Innsbruck

Pre-registered on OSF ↗

Date01 July 2026

VenueWCERE 2026

Projectmanydesignscarbon.online

Funded by

Motivation02

Effective carbon pricing, limited public support

A price on carbon is widely judged effective and cost-efficient (Nordhaus 1992; Stechemesser et al. 2024), yet only 28% of global emissions are priced — at ~$19/t against a social cost near $120/t (World Bank 2025; EPA 2023).
The binding constraint is public and political support, not the instrument itself.
Prior evidence on the effectiveness of nudges in altering behavior raises concerns about publication bias and backfiring effects (DellaVigna & Linos 2022; Maier et al. 2022; Mertens et al. 2022).
Most evidence focuses on (pro-environmental) behavior (e.g., Allcott 2011; Tiefenbeck et al. 2019); support for system-level policy, measured with real-world outcomes, remains largely unexplored (see Vlasceanue et al. 2024 and Dechezleprêtre et al. 2025 for stated support measures).

The question

“What is the impact of behavioral interventions on real-world support for a price of carbon?”

The Many-Designs Approach03

One question, many designs

Instead of “one team, one design, one dataset,” the whole field designs the same experiment — run all at once (Huber et al. 2023).

Diverse by design: many teams, randomly drawn from a pool of applicants (economics, psychology, and the behavioral sciences broadly), each contribute an independent RCT answering the exact same research question.
Clean quantification of design heterogeneity: a shared, simultaneously collected participant pool and one pre-registered analysis, so differences reflect the designs, not the sample or the statistics (Holzmeister et al. 2024).
Open Science as a living principle: pre-registrations, protocols, code and outcomes of all designs are shared and reported (OSF repository ↗), regardless of the effect produced, mitigating publication bias.
Taken together: highly-powered and field-capturing meta-scientific evidence.

Each team's task

Design an intervention and a control condition, and propose an observed real-world outcome that measures support for carbon pricing.

135teams applied

55designs run

19,558US participants

110conditions (55×2)

Experimental Procedure04

Experimental protocol

From 55 studies to one estimate05

Methodology: standardize, then meta-analyze

Figure 1 · What the teams built06

The design space the crowd produced

Figure 3 · The whole distribution of results07

Expectations versus observed effects across 55 designs

Figure S6 · Expectations vs reality08

Observed effects versus researchers' expectations

Figure 2 · Pooled effects09

Very small, positive, and consistent across (primary) outcomes

Figure 4 · What explains the variation?10

Difficulties in explaining variation in effects

Robustness11

Robust across specifications, subsets, and single designs

Implications12

Small and over-assessed effects?

What we find

Small but statistically significant meta-effects (Cohen's d ≈ 0.04–0.08; ~1–2.4 pp); at the study level most effects are indistinguishable from zero.
Teams themselves and their peers were systematically overconfident about the effects (see e.g., Menkveld et al. 2023; König-Kersting et al. 2025).

Why are effects smaller than reported in prior work?

Preferences on carbon pricing may be rigid and not easily moved (Mildenberger et al. 2022; Dechezleprêtre et al. 2025).
Interventions may be less effective for system-level policy than for individual behaviour (Bergquist et al. 2023; Sinclair et al. 2025).
Prior estimates are likely inflated by publication bias — which may also anchor researchers' miscalibrated beliefs (Scheel et al. 2021; Borjas & Breznau 2026).

0.063full-sample d

0.26if only “sig.” reported

Reporting only the significant studies would inflate the pooled estimate roughly fourfold.

Limitations13

Limitations

Some real-world proxies may carry mechanical features; conservatively excluding 7 still leaves a detectable effect (d = 0.049, k = 48).
Inference rests on the meta-effect — individual studies are low-powered for effects this small.
55 of 135 teams were run, randomly selected to avoid selection bias.
US-only sample — cross-cultural and cross-context generalization remains open.

Takeaway

Behavioral interventions raise support for carbon pricing — but only very modestly, and, importantly, by less than researchers expect.
The many-designs approach aggregates independent studies into one collectively and transparently reported estimate (alongside the non-negligible heterogeneity of on-par designs).

manydesignscarbon.online
interactive results & all 55 designs

Thank you · 108 co-authors across 55 research teams · University of Innsbruck · Funded by the Austrian Science Fund (FWF), SFB F63 (10.55776/F63)

The collaboration—

55 research teams · 108 co-authors

References—

Fifty-five Crowd-Sourced DesignsAssessing Carbon Pricing Support