Choose timezone
Your profile timezone:
CIDeR-ML General Meeting
→
America/Los_Angeles
Description
https://u-tokyo-ac-jp.zoom.us/j/83932834349
Recording
Minutes:
Quick recap
The meeting focused on reviewing Zhenxiong's progress on optical SIREN prediction analysis, with discussions about bias in visibility plots and post-tuning results. Zhenxiong presented plots comparing pre-tune and post-tune performance, but discrepancies were identified between current and previous results, particularly regarding bias at higher charge levels, and post-tuning becoming worse. The team discussed potential issues with sample splitting for training and validation, as well as concerns about the Cherenkov profile implementation. Patrick suggested consulting with Ryo about updating the Cherenkov profile and recommended investigating different sampling schemes to understand and potentially correct the introduced bias in post-tuning results.
Next steps
- Zhenxiong: Double check the samples and plotting code to investigate the inconsistency in bias results between last week and this week, especially regarding the handling of 0 PE events and sample labeling.
- Zhenxiong: Split the dataset into independent training and validation samples and generate validation curves to check for overtraining effects.
- Zhenxiong: Investigate and try to understand why the post-tune results show increased bias at high charge, including testing different sampling schemes for track generation as suggested.
- Zhenxiong: Double check how the Cherenkov profile is updated and used in the prediction, and consult with Ryo and the LUCiD Group regarding their methods for generating Cherenkov profile SIRENs.
- All participants: Review and finalize goals for the upcoming April workshop in the following weeks, including review of project status and goals for the workshop.
Summary
April Meeting Logistics Planning
Patrick discussed logistics for the upcoming April meeting, reminding attendees to check and book their hotel accommodations. He also mentioned that the next workshop is about a month away and encouraged the team to start planning goals based on the previous workshop's discussions. Patrick suggested reviewing project status and goals in the coming weeks to prepare for the next workshop.
Photon Shotgun Visibility Analysis Results
Zhenxiong presented plots comparing Photon Shotgun visibility and optic SIREN prediction, finding similar results to Ka Ming's previous work. She also identified a tail of bias for the first bin, which Patrick de noted he hadn't noticed in previous plots. The team discussed why this bias wasn't visible in the publicly shared plots, though the reason remained unclear.
Data Plot Bias Discussion
The team discussed biases in plots, particularly focusing on the visibility of data at low energy bins. Zhenxiong explained that she had removed zero PE events for both observed and predicted data, which PatrickD noted should not have affected the results significantly since there were no zero PE events at higher charges. Patrick Tsang suggested that the left-hand side of the plot looked acceptable as it used MC information, but expressed confusion about the right-hand side after post-tuning, expecting worse results than the pre-tune.
Plot Inconsistency Discussion Meeting
Patrick Tsang and Zhenxiong discussed inconsistencies in a plot, particularly at the 40 PE mark where the red dot was nearly zero, which Patrick noted was unusual. Patrick de suggested double-checking the samples and plotting process, as the differences from the previous week were significant. Zhenxiong confirmed she had training loss graphs and explained she used a fraction of the samples for validation, which Patrick acknowledged.
Data Bias and Overfitting Analysis
The team discussed issues with biased numbers and variance in their data analysis. Patrick Tsang suggested splitting the training and validation samples to check for overfitting, while PatrickD recommended investigating how the bias affects the reconstruction pipeline. They agreed to explore different sampling schemes to determine if changes are needed, particularly regarding the current 0.1cm sampling and fixed 1 GeV assumption. The team also planned to examine why the post-tune introduced bias compared to the pre-tune.
Cherenkov Profile Yield Discussion
PatrickD discussed a light yield problem related to the Cherenkov profile limitation and asked Zhenxiong to double-check how to update the profile for prediction. Zhenxiong agreed to consult with Ryo and mentioned that LUCiD Group has its own method for generating Cherenkov profiles.
