Project Logo

#ManyDesignsCarbon

#ManyDesignsCarbon

A Crowd-Science Project

About the project

#ManyDesignsCarbon is an innovative crowd-science initiative focused on exploring the influence of behavioral interventions on the social support for carbon pricing.

This project unites research teams from across the globe, ranging from PhD students to full professors, to develop and propose experimental designs. Each team of up to two researchers will address a critical question:

What is the impact of behavioral interventions on the real-world support for a price of carbon?

The spirit of the project is to tackle the introduction or substantial increase in the price of carbon, aiming to advance towards social carbon pricing. Following the U.S. Environmental Protection Agency (2023) a lower-bound estimate of the carbon price that incorporates all social costs is $120 per tonne of CO2, which would equal about $1.00 per gallon of gasoline.

Project coordinators will jointly analyze the results from around 40 proposed designs to develop a comprehensive metascience paper, co-authored by all participants whose designs were implemented. The study will highlight variations in design choices and their results, accelerating knowledge generation within the field by years as dozens of designs are developed simultaneously instead of partly sequentially.

Procedure

Participation in #ManyDesignsCarbon involves the following 6 steps: Sign-up, Design Proposal Submission, Filling out two Surveys (A and B), Programming of the Experiment, and Manuscript Writing (optional).

  1. Registration
    1. Research teams (RTs) register to participate (without initially submitting a research design).
    2. Project coordinators then screen these applications for eligibility according to the project's eligibility criteria and invite compliant RTs to outline their design proposals in Step 2 through a pre-registration.
    3. If total registrations exceed 42, a pre-registered random selection will be made to choose 42 research teams to submit their designs.

  2. Design submission and selection procedure
    1. Eligible RTs submit a detailed pre-registration of their research proposals and complete Survey A on their design's expected effect size (Cohen’s d or Cohen’s h).
    2. Project coordinators pre-screen all submitted designs for adherence with the design conditions.

  3. Survey B
    4. RTs are required to evaluate ten other design proposals. Specifically, for each of the designs, RTs will be asked
    • how informative the proposed research design is to answer the following research question (Likert scale from 0 to 10): "What is the impact of behavioral interventions on real-world support for a price of carbon?"
    • to classify the type of behavioral intervention employed (open question).
    • predict the standardized effect size of the design (Cohen's d or Cohen's h depending on the dependent variable).

  4. Experimental implementation
    • Selected designs are implemented as online experiments on Prolific following this experimental protocol, and funded by the project coordinators (i.e., 350 observations per implemented design; 14.000 observations expected for the entire project).
    • RTs are responsible for programming and hosting the experimental software (including instructions) on their servers.

  5. Analysis
    • RTs must specify that main treatment effects can be estimated using Ordinary Least Squares (OLS) or a logit model, with the dependent variable being real-world support for a price of carbon.
    • Upon completion, project coordinators download the raw data to which they must be granted access by the RTs, who also need to provide a detailed codebook of all data points.
    • Project coordinators will analyze the data as specified in the project coordinators' pre-analysis plan.

  6. Project coordinators will write a metascience paper; all members of selected RTs will be co-authors.

Project coordinators. The project will be coordinated by researchers from the University of Innsbruck. Meet the project coordinators.

Pre-analysis plan. A detailed Pre-Analysis Plan (PAP) was registered with a time stamp on the Open Science Framework on May 15, 2024. An addendum specifying standardized analysis methods and exact hypotheses will follow after design registration but before data collection commences. The entire OSF repository with the pre-registered analysis plan of the project, its addendum, the pre-analysis plans of the RTs (see Pre-registration), the data and the experimental software packages will be made publicly available after the publication of the paper.

Questions. We have set up a detailed FAQ with commonly asked questions. Of course, feel free to contact us via mail.

Why participate?

  • Joining #ManyDesignsCarbon offers more than just engagement in a groundbreaking project. It satisfies your academic curiosity with the opportunity to explore a significant and timely research question.
  • For designs that are chosen for implementation, the project coordinators will fund Prolific participant payments, alleviating financial concerns and allowing you to focus on research excellence.
  • If your design is selected, you will gain co-authorship on a paper slated for submission to a leading scientific journal. This follows in the footsteps of a previous successful project by two of our project coordinators on "Competition and Moral Behavior", published in PNAS in 2023 (refer to Huber et al., 2023).

Eligibility

To participate in this project, research teams (RTs) must meet the following eligibility criteria:

  • each RT may consist of no more than two members.
  • at least one team member must hold a PhD in Economics, Psychology, or a related field.
  • at least one team member must have conducted an experimental study that is published, accepted for publication, or documented as a working paper.
  • each researcher may only participate in one RT.

Cases that meet borderline criteria will be decided through a majority vote by the project coordinators.

Experimental protocol

provided by project coordinators provided by research teams INTRO BEHAVIORAL INTERVENTION CONTROL REAL LIFE CARBON PRICING SUPPORT POLICY SURVEY END

The protocol for our experiment is outlined in the diagram above. Components in white are managed by the project Coordinators, while the green parts represent the contributions from selected Research Teams (RTs). The entire experiment is designed to take approximately 15 minutes, with 7-8 minutes dedicated to the intervention.

INTRO: Participants on Prolific begin with a comprehensive introduction that includes consent procedures and a CAPTCHA task to ensure genuine responses by preventing bots.

INTERVENTION: Following the introduction, Prolific participants are randomly assigned to one of two conditions—BEHAVIORAL INTERVENTION or CONTROL—across the 42 selected designs. This phase is followed by the measurement of real-world support for a price of carbon, as outlined by the RTs in their proposals. ''Real-world support'' implies that actions of the Prolific participants during the policy-support measurement stage must have consequences that extend beyond the experimental context. All designs must meet the specified design conditions.

POLICY SURVEY: Prolific participants then complete a follow-up questionnaire (policy survey) provided by the project coordinators. RTs integrate this questionnaire into their software (either oTree or Qualtrics), which will be provided by the project coordinators. Specific questions are detailed in our registered pre-analysis plan. The repository including the pre-analysis plan will be made public upon publication of the paper.

END: Having answered the policy survey, Prolific participants are at the end of the experiment and will get their payout (i.e., flat fee of 1.50 GBP plus a potential bonus of max. 3 GBP on average; the latter is up to the RTs).

Design conditions

To be included in this project, the submitted pre-registration has to adhere to the following design conditions:

  • Each design is implemented as an online experiment to be run on Prolific with 350 observations.
    • The experiment needs to be suitable for administration on Prolific.
    • Designs must adhere to Prolific's terms & conditions for researchers.
    • Interventions should not exceed 7-8 minutes, inclusive of dependent variable elicitation (RTs chosen real-world measure of support for a price of carbon).
  • Experiments must employ a between-subjects design with two conditions:
    • A control condition without intervention (CONTROL).
    • A treatment condition with intervention (POLICY).
    • RTs must ensure the possibility to generate two distinct links for each condition on either oTree or Qualtrics (project coordinators must be granted access to the experiments to generate the links).
  • RTs need to ensure incentive compatible payments that should at least cover the participants' opportunity costs and adhere to Prolific’s minimum payment guideline of GBP 6 per hour without imposing negative earnings.
  • RTs must define a clear metric for real-world support for a price of carbon.
  • RTs must avoid deception of subjects.
  • RTs must ensure subject anonymity within the experiment
  • RTs must avoid measurements of physical state (e.g., saliva samples, blood samples) and no physical or psychological harm.
  • RTs need to provide clear information on experiment duration, repetitions, interactions, and randomization procedures.

All proposals are subject to an initial review by the project coordinators to ensure compliance with these standards. Proposals meeting borderline criteria will be decided through a majority vote by the project coordinators.