GRIDmeter
  • GRIDmeter™
  • Methods
    • GRIDmeter
    • FLEXmeter
  • Code
  • Blog

How Stable are Comparison Groups over Time?

8/31/2020

1 Comment

 
Picture
●Would a comparison group that matches the baseline usage pattern of a treatment group start to exhibit bias over time that would negate some of the savings of the treatment group?
●We know that there might be self-selection issues that might need to be addressed during the selection of the comparison group
●But we don’t know what the variability of the bias might be and we don’t know whether or not it might get worse under conditions of large-scale energy changes, such as during COVID

experiment

●Simulate the selection of treatment and comparison groups for a program
●Target a subset of high peak load users, similar to what might done for a program that is focused on peak load reduction
●Select a variety of sizes of “treatment” and “comparison” groups from the larger targeted population that can be compared against each other

pure doppelganger   vs   two smallest groups   vs   two largest groups

Picture
Picture
Picture

takeaways:

 Bias is inevitable in comparison groups due to factors beyond our ability to control for

2) Larger groups will reduce bias, but treatment group may still diverge from comparison group over time

3) Need for more systematic approach to determine optimal comparison and treatment group sizes

Standardizing Stratification

Picture
Stratification is about binning

●What parameters?
●Equal distance or population?
●How many bins?
●How many parameters?
●In what order of priority?
●Should there be an order?
●How do you gauge success?


Stratified Sampling: What are the Constraints?
The only option in resampling is the elimination of meters.

Two main constraints:

1.The size of the comparison pool
2.The number of meters needed in the final sample

Picture
Standardizing Stratification: Dealing with Multiple Parameters

Multiple parameters can generate a more representative sample

●Complexity: Stratifying the first parameter and then moving on to the next will inherently change the first
               ○Solution = Simultaneous 2D or nD binning
●How should the individual parameters be prioritized and binned?
               ○Solution = An optimal scheme should be determined via                    algorithm.
●How do you gauge success?


Video of the August 28, 2020 Meeting: Comparison WG VIDEO - 2020-08-28
Slides from the August 28, 2020 Meeting (cumulative): Comparison WG SLIDES - 2020-08-28
Chat Record from the August 28, 2020 Meeting: Comparison WG Chat - 2020-08-28

1 Comment
California Black Girls link
11/4/2022 05:24:50 pm

Greeat read thankyou

Reply



Leave a Reply.

    Archives

    December 2020
    October 2020
    September 2020
    August 2020

    Categories

    All

    RSS Feed


Contact the Project Team
©2020 Recurve Analytics Inc. "Recurve" is a trademark of Recurve Analytics Inc. All rights reserved.
​Privacy Policy. Terms of Service.
Picture
  • GRIDmeter™
  • Methods
    • GRIDmeter
    • FLEXmeter
  • Code
  • Blog