Steps
- Open an alarm (or its group).
- Turn on Dismiss notifications.
- Tap Save.
Matching notifications are now removed from your panel after the alarm handles them.
To use it fully
Set the global rule in Settings > Notifications > Dismiss Notifications to Keep unless set to dismiss by alarm, so only alarms that ask for it dismiss notifications.