Constraining Lepton Flavor Violating Higgs Couplings at the HL-LHC in the Vector Boson Fusion Channel

Abstract

We explore the parameter space of lepton flavor violating (LFV) neutral Higgs Yukawa couplings with the muon and tau leptons that can be probed at the high-luminosity Large Hadron Collider (HL-LHC) via the vector boson fusion~(VBF) Higgs production process. Our projected sensitivities for the Standard Model Higgs ($h$) LFV branching ratio $\mathrm{Br}(h\to \mu \tau )$ in the $pp\to hjj\to (h\to \mu \tau )jj$ channel at the HL-LHC are contrasted with the current and future low-energy constraints from the anomalous magnetic moment and electric dipole moment of the muon, as well as with other LFV observables, such as $\tau \to 3\mu$ and $\tau \to \mu \gamma$. We also study the LFV prospects of a generic beyond the Standard Model neutral Higgs boson ($H$) with a mass in the range of $m_H\in [20,800]$GeV and give the projected model-independent upper limits on the VBF production cross-section of $Hjj$ times the branching ratio of $H\to \mu \tau$ at the HL-LHC. We interpret these results in the context of a two-Higgs doublet model as a case study.

Comments: 16 pages, 8 figures, 6 tables. Revised version, to appear in Phys. Rev. D