comparison group principles: june 26, 2020

why comparison groups?

• If COVID were a time-bound, uniform event with a clear starting and stopping point, we might choose to pursue other, more familiar NRE strategies
  
  
  • Removing COVID-period meter data from consideration in modeling/estimating savings; applying a discount to savings during the COVID period; creating a model that interpolates savings for COVID period
    
    

• Because COVID has led to non-routine changes in energy use in virtually every building in the world; because the recovery from COVID will take months, if not years - Our only choice is to track changing patterns of consumption in the population and apply those trends to our treated customers

proposed methodology to test

1. Prior to procurement, establish program eligibility rules and apply equally to participants and non-participant comparison group
   
   
2. Select program-appropriate parameters for establishing comparison group through stratified sampling of eligible population
3. Sample from parameterized matrix according to distribution weights so that a comparison group of sufficient size is created that reflects anticipated program enrollments
4. Track changes in consumption historically to establish variance
5. Track changes in consumption going forward to determine program savings
   
   
   1. So that treated customers and non-treated customers are evaluated using the same set of baseline conditions, create monthly vintages of the comparison group that align with enrollment dates amongst treated customers

Video of the June 26, 2020 Meeting: Comparison WG VIDEO - 2020-06-26

Slides from the June 26, 2020 Meeting (cumulative): Comparison WG SLIDES - 2020-06-26

Chat Record from the June 26, 2020 Meeting: Comparison WG Chat - 2020-06-26