When a team says it gets too many alerts, the instinct is to delete monitors or raise thresholds. Usually the alerts are accurate; they are just landing in front of people who cannot act on them.
Route by owner and urgency
Send each alert to the team that owns the service, and decide per priority how loud it should be. A failed nightly report can wait in a channel until morning; a checkout outage should page someone now.
Review what woke people up
Once a month, look at every alert that paged someone outside working hours. For each one ask: did it need a human, and was it the right human? Change the routing rules, not the monitors, when the answer is no.