Blinding and unblinding analysesBlinding and unblinding analyses achintya Thu, 06/28/2012 - 12:30
CMS performs searches for new particles by looking for signals amidst a background of known physics. If the data begin to indicate something more interesting than merely background – for instance, more decay events than expected in a certain region – it is important to make to make sure that the observation is statistically significant by collecting and analysing more data. “There is however a human tendency, sometimes at a subconscious level, to optimise one’s analysis based on what is already seen,” says Albert De Roeck, co-convener for the CMS Higgs group. To avoid such bias while analysing new data, physicists draw “blinds” over the region where an excess of decay events is expected; this region is only “unblinded” when they are satisfied with their procedures. This ensures objectivity when it comes to looking for much-sought-after signs of new physics, and gives confidence in the ultimate result. The procedure is similar to that used by medical researchers when testing a new treatment. For example, in the search for the Higgs boson, CMS ruled out the presence of the Standard Model Higgs boson in the range of 127 – 600 GeV at 95% confidence level using data collected up to the end of 2011. At the same time, a modest excess of events was observed at around 125 GeV. The accompanying figure shows a close-up of the result of the search where the Higgs boson decays into two Z bosons, which then decay into two leptons each. The region that was not ruled out in 2011 as well as its immediate surroundings (110 – 140 GeV) were blinded during analysis of new data (the vertical band in the figure) — notice the absence of data points here. The rest of the region was used to understand backgrounds and fine-tune the analysis. Also shown in red is the simulated signal for a Higgs boson with mass 126 GeV. All of the available data, including those in the blinded region, are used in the analysis; you just don’t peek at this region to see how it changes as you are tweak various selection criteria. In many of the on-going searches, data corresponding to a possible signal look very similar to data corresponding to background. Therefore, once the analysis is blinded, physicists must optimise two things: 1) simulations, to help determine the criteria for selecting signals and 2) the methods for rejecting or quantifying background events using data from the region where there is no excess. To cross-check all the details, the various steps involved in the analysis procedures are usually carried out by at least two independent teams. If both teams see similar results in the background region, the analysis is signed off by the wider collaboration. It is only after the sign-off that the signal region is unblinded. The results of the unblinding are put through further scrutiny by the collaboration before being are made public.