ClusterQA
Last updated
Last updated
This part of the QA must be run for any calorimeter related analysis (for EMCal/PHOS/DCal and hybrid analyses PCM-EMCal, PCM-PHOS, PCM-DCal)
For bad cell identification, we need the output from GammaCalo with cellQA output contained (perferably option '5') downloaded runwise (if you don't know how to download runwise output, look here Download Output).
1. Run QA_RunwiseV2.C with doEventQA = kTRUE and doClusterQA = kTRUE (doMergedQA for EMCal merged analysis) with doExtQA == 2! (important for cell level output) using data + MC input in order to generate all needed histograms + txt-files for bad cell identification
Important example histograms for the runwise QA and bad cell identification (plots are from the QA stage and include cells which were declared as bad in the meantime):
Number of Clusters per event
Cluster eta/phi maps for each run for data and MC - a difference can be clearly seen which MUST be corrected in MC (switch off the given RCUs for MC)
2. CellQA step: identify dead and warm/hot cells comparing data and MC information Have a look at the histograms created in $CUTSTRING/$ENERGY/$SUFFIX/Runwise/$PERIOD/ExtQA/* and especially .../ExtQA/MissingCells/* for dead cells, so you get a feel for the variables you are looking at. Dead/warm/hot cell identification is done by comparing the cells EFrac to mean EFrac of neighboring cells. EFrac is defined as follows:
EFrac = cells energy fraction of full cluster energy, summed over all events --> turned out to be a much better discriminator than just looking how often cells fired
An example of the EFrac plotted versus CellID which is generated by the RunwiseQA:
We have a dead cell candidate:
if((nCurrentEFracmean/3 && mean80) || (nCurrentEFracmean/5 && mean40 && mean80) || (nCurrentEFracmean/8 && mean10 && mean40) || (nCurrentEFracmean/10 && mean10))
A warm/hot cell candidate:
if((nCurrentEFrac2_mean && nCurrentEFrac_80) || (nCurrentEFrac3_mean && nCurrentEFrac20) || (nCurrentEFrac4_mean && nCurrentEFrac8) || (nCurrentEFrac_5_mean && nCurrentEFrac5))
Above conditions for dead/warm/hot cell candidate identification are being applied in function CheckHotAndColdCellsEFracRunwise in QA.h! Be careful that a minimum number of events is needed to process the run in this function (runs with ultra low statistics are excluded as it is impossible to look for hot/dead cells besides the obvious candidates which should then appear in other runs as well) The resulting dead/warm/hot cells are written to respective *.log files that can be found in the output folder.
(located in $CUTSTRING/$ENERGY/$SUFFIX/Runwise/$PERIOD/HotCells_FiredInNRuns_)
Example how the HotCellsRunwise looks like:
Example how the DeadCellsRunwise looks like:
The next steps 3./4. are needed as by statistical fluctuations some cells can fire more often or less often than usual, therefore we need to apply further conditions for dead/hot cell identification:
3. Check Dead-Cells for different runs via ClusterQA_DeadCellCompareV2.C
This macro can be run using by enabling the 'doCellQASummary' as last parameter in the QAV2.C, by default this parameter is set to kFALSE. It will then check if the config-file contains the bare minimum of settings to run the ClusterQA_DeadCellCompareV2.C and proceed with the running if this is the case. Thus you will need to add the following lines to your configuration file:
If you want to run the macro without starting QAV2.C you have to create a new configuration file which contains the full set of parameters needed, like configDeadCellQA_LHC16qt_PHOS.txt. In that case it can be started with:
The determination of dead cells is done using the for-loop around line 364. Different decisions when to consider dead cells:
if cell is dead cell candidate in consecutive number of runs (currently set up with 4),
if at least 10 dead cell candidates are found with in the same run range or cell candidate is identified to be dead in a given % of analysed runs, represented by fractionThesh.
All results are written/summarized in log-files, which will be not located in the cut folder, but in a folder called ClusterQA_DeadCellCompare.
Example $DATASETNAME-detailed.log:
and according to the selection criteria the final list of dead cells ($DATASETNAME-final.log):
In addition, there will also be an output file called $DATASETNAME-Runwise.log. The information included in this log file will be the $CellID-$RunNumber-$Sigma. Note for all cold cells the sigma value is set to 10. This will later be used for plotting in ClusterQA_CellCompareV2.C. An example of this can be found below.
4. Check Hot-Cells for different runs via ClusterQA_HotCellCompareV2.C
As for the previous step, this macro can be run by enabling the 'doCellQASummary' as last parameter in the QAV2.C. It will then check if the config-file contains the bare minimum of settings to run the ClusterQA_HotCellCompareV2.C and proceed with the running if this is the case. Thus you will need to add the following lines to your configuration file:
If you want to run the macro without starting QAV2.C you have to create a new configuration file which contains the full set of parameters needed, like configHotCellQA_LHC16qt_PHOS.txt. The macro can then be started with the following command:
Determination of warm/hot cells using for-loop around line 248 via 'threshNFired' and 'threshNTotalFired' for 3. + 4. produce output log-files which summarize the bad cells to be excluded in OADB from runwise dead/warm/hot cell determination.
All hot cell candidates are written/summarized in log-files, which will be not located in the cut folder, but in a folder called ClusterQA_HotCellCompare.
Example $DATASETNAME-detailed.log:
and according to the selection criteria the final list of hot cells (sorted by run or sorted by cell ID) in ($DATASETNAME-final.log):
(cell ID / run number)
Again, there will also be a $DATASETNAME-Runwise.log for the hot cells. The only difference from the cold cells is that instead of getting a default value of 10, the cell will have a decimal sigma value. The format still goes as $CellID-$RunNumber-$Sigma. For clarity, an example is included below.
After identifying dead/hot cells on runwise level, the general periodwise level CellQA + ClusterQA step follows.
5. CellQA step on periodwise level via running of ClusterQA.C Identify bad cells on periodwise level by using data and MC information. In order to run QA on a periodwise level, a separate config file will need to be created. As before, example configurations can be found in the TaskQA/ExampleConfigurations folder. Additionally, a merged root file from all good runs in a period will be needed for this step.
Setting for bad cell identification are defined in QAV2.C (Double_t arrQAEnergy[4]; Double_t arrQATime[4]; Double_t arrQAHotCells1D[4]; Double_t arrmin2D[9]; Double_t arrmax2D[9];)
The cell IDs are also written to the log-file which is generated:
In the event that the program crashes at this step (gives error message: ERROR: allCells.size() too big 82442, check cuts! RETURNING…,) then one must revise the cuts given in the config file. For reference, below are the cuts and which figure they correspond to. Note that visually, cut values will be represented as dotted lines on the existing plots.
min2D, max2D
CellHotCells2D_$DATASETNAME.eps
setQAEnergy
CellEnergyVsSigma_$DATASETNAME.eps
setQATime
CellTimeVsSigma_$DATASETNAME.eps
setQAHotCells1D
CellHotCellsRescaled_$DATASETNAME.eps and CellHotCellsTime1D_$DATASETNAME.eps
An overview plot is generated with an overview by which cut a cell has been identified as bad candidate (the cell IDs are also stored in log-files):
Furthermore, the energy and time distributions are plotted for data and MC:
Real bad cell candidates are immediately visible in comparison to the rest of candidates.
In addition, the energy distribution of each cell candidate is compared between data and MC (normalized to the number of events) to help with the judgement if cells are really bad. At the same time the cell time distribution is compared to that of a random good cell:
Compare data/MC energy and time distributions for all bad cell candidates found in 5. Also use 8.
Then, merge bad cell candidate list from 5. with lists found in 3. and 4. to be excluded in OADB
6. Run bash file lookAtCellQa.sh to sort the energy distribution plots.
This script is simply to make the sorting of bad cells, maybe bad cells, and okay cells based on the energy distribution plots easier for the user. The macro takes 3 parameters, the location of the energy distribution plots, the location of the ok/maybe/bad folders (will be created by the macro), and the suffix (i.e. "png").
The script will then guide the user through the process of sorting, first by showing examples of good cell plots for the data set so that the user can get a feel on how to sort future plots. Then the user can decide via the keys 1,2,or 3 in order to move these plots into the corresponding ok/maybe/bad folder.
7. Run ClusterQA_CleanCellLogs.C via QAV2.C
This macro will go through the log files in ClusterQA_HotCellCompare and ClusterQA_DeadCellCompare and check if some of the cells in there were already flagged as good by the user (in the process described in Step 6) and will delete them. Cells flagged as good will be removed from the log file instead of adding the ones you flagged as bad in Step 6, because we have more confidence in the distributions flagged as good (because the eye is usally better at spotting good things). The cleaned logs will be written to $OriginalFile-Cleaned.log. The information for which folders are flagged as good (and maybe) by the user are taken from the folders containing the Cell#_EnergyComparison files.
Below is the usage to be added to the config file to enable the ClusterQA_CleanCellLogs.C.
8. Run ClusterQA_CellCompareV2.C via QAV2.C to visualize bad cell candidates found in step 5.
This can be run from the steering macro QAV2.C with 'doCellQASummary' (the last parameter in QAV2.C) set to kTRUE. Additionally the following lines must be added to the the configuration file. These additions along with corresponding explanations are shown below.
If you are sure you wanna flag certain cells, which you identified as bad in step 6 and which are not contained in the automatically generated output, please add a list of cells using the variable CellCompareManualBadChannels in the configuration file. The corresponding list should be given as text file in which the cells are separated by line feeds. Cells which are listed in that file will be marked as bad for the full range of runs given in the configuration regardless of the input provided from different sources, thus they will appear as green row in the plots. Please remember to repeat step 5 & 6 for each run block you choose to group and only merge the runs of that block into the corresponding file and repeat the manual flagging. As otherwise you might end up accidentally flagging to many cells in certain run blocks.
In addition to the output plots, which will be described below. This macro also outputs two additional log files for every data set. Depending on the value of runRange this log file will either contain a list of the bad cells within a range of runs specified by runStart and runEnd or a list of all the bad cells in a data set. In the case where runRange is set to 1 the log file will be named BadCells[Cleaned]$DataSetName_**$runStart__$runEnd.log and in the case where runRange is set to 0 the log file will be named BadCells[Cleaned]_$DataSetName.log**. An example of the output for either of these log files is shown below.
From running ClusterQA_CellCompareV2.C there will be two types of histograms generated; one runwise plot of Cell ID (for bad cells) vs. run number and another period wise plot of Cell ID (for bad cells) vs. period. Examples of these plot are shown below. For the runwise plot with lines indicating the cells which would not fullfil the CellCompareMinAverageSigma-condition is also provided. In the corresponding plot the cells which would not be marked as bad are crossed out with a red dashed-dotted line.
The above histogram should be used in to aid pattern recognition in order to determine if cells should be masked globally, or within a certain run range. The idea is that using run ranges would increase statistics. Note that here both hot and dead cells are shown in combination and the z axis represents the sigma.
DISCLAIMER: This is a summary and not a standalone guide. Please always have a look at all the steps described previously, before doing the QA!
Make sure you have the runwise output of your task, aswell as the merged output for periodwise QA available.
Run QA_RunwiseV2.C with doEventQA = kTRUE and doClusterQA = kTRUE (doMergedQA for EMCal merged analysis) with doExtQA == 2.
An example configurations can be found in the ExampleConfigurations folder. E.g. your config file could look like this ( please remember to make sure, that you use only tabs or spaces in your config file! not a mix of both):
Run the QA on a periodwise level using QAV2.C. The example config file given below, contains the minimum settings needed to run on the periodwise level (in this case only for period LHC10b), aswell as running the hot cell compare macro and the dead cell compare macro. You should probably create one config file per period you are analysing, and run the QAV2 macro for each period seperately to avoid any complications.
After running over the periodwise output for the first time, you will probably get an ERROR message. This means you have to adjust the cuts in the enableCellQA block in the config file. When the cuts are correct, continue!
Run bash file lookAtCellQa.sh to sort the energy distribution plots. This will take some time, but it is worth the effort! Don't get too frustrated.
Rerun the QAV2.C macro over the period you just sorted in the previous step, but this time add the lines needed for the ClusterQA_CleanCellLogs.C macro. Have a careful look at its output, especially at the cells you flagged as bad, but the macro didn't.
Rerun QAV2.C with the settings and the lines in the config file needed to run run ClusterQA_CellCompareV2.C via QAV2.C .Redo this step for the different subranges you slected after looking at the overview plots and remember to provide it with a file containing the cells you think should be flagged as bad, even though the compare macros didn't evaluate them as bad in the previous step.
You should now have different txt files containing bad cells for different runranges. Remerge the periodwise output according to those runlists and redo the periodwise QA in this range.
Repeat for each period
You are now done with the first iteration of the QA. Re run the task using this bad channel map and then redo QA to make sure you didn't miss any bad cells in first iteration.
The cluster QA covers general cluster+cell properties like:
generated histograms (list of examples) (full set of generated histograms can be deduced from the macros themselves/or from the output generated):
Cell histograms (number of cells fired, cell energy/timing,...)
Cluster histograms (number of clusters per event, eta/pi distributions,...)
Running the ClusterQA(_Runwise).C will save the output into the following folder structure:
CUTNUMBER/SYSTEM/EventQA/
In addition, *.root files will be generated in CUTNUMBER/SYSTEM/ containing all the histograms as well.
9. Runwise ClusterQA step by analysing output from ClusterQA_Runwise.C Carefully check all output from runwise histograms with special focus on data/MC comparison (Is the MC able to reproduce all QA histograms extracted from data? Does the MC follow the trends seen in data? Are there any suspicious runs or any observations that cannot be explained?...)
10. ClusterQA step by analysing output from ClusterQA.C Carefully check all output from histograms with special focus on data/MC comparison (Is the MC able to reproduce all QA histograms extracted from data? Does the MC follow the distributions seen in data? Are there any suspicious observations or is there anything that cannot be explained?...)
10. Optional: run ClusterQACompare (needs to be configured within macro - not _yet included in steering macros) to compare different data sets (MB vs calorimeter trigger for example for a given dataset; needs input from runwiseQA and periodwiseQA)
Carefully check all output from runwise/full output with special focus on data/MC comparison (Is the MC able to reproduce all QA histograms extracted from data? Does the MC follow the trends seen in data? Are there any suspicious runs or any observations that cannot be explained?...) In general, they should be stable vs. run number - however, one of the exceptions is pileup which may vary from run to run -> need to be taken with special care!
There is an existing option to write specific clusters to a txt-file with the GammaCalo task (for example cluster combinations with a very small opening angle or select especially huge clusters,...). Once you produced the txt-files with help of the GammaCalo task, you can visualize the clusters by help of the macro Grid_PlotGammaCaloDebug which can be found in TaskQA.
It can be called via:
Grid_PlotGammaCaloDebug(TString filePath = "/home/daniel/data/work/debugOutput.txt", TString outputDir = "DebugPlots", Bool_t plotFullEMCal = kFALSE, Int_t debugOption = 0)
The hot cell candidates are summarized in such plots (How often does a cell fire per run and how many hot cell candidates are available per run):
An overview plot is generated with an full overview in which period a cell has been identified as bad candidate, if more than one period had been handed to the macro:
Cut overview histograms (vs. )
