Notification delays [CA Region Only]

Incident Report for Hypercare

Postmortem

What Happened?

At 18:50 EST, our monitoring system flagged a potential issue with the notification service in Hypercare. An immediate investigation was launched by the team. Testing concluded the issue was isolated to users based in Canada.

Impact

The notification service interruption affected several critical functionalities within Hypercare, including: delivery of Hypercare message notifications within the platform, outgoing pages and SMS sent through Hypercare and integrations with some third-party services, such as Everbridge. The investigation indicated that messages sent between 18:34 and 19:05 EST experienced prolonged delays of up to 20 minutes in notification delivery.

Resolution and Next Steps

The issue was resolved at 19:05 EST. Our team is continuing to monitor the notification service and conducting further investigation on the root cause.

Posted Feb 20, 2025 - 10:14 EST

Resolved

This incident has been resolved.
Posted Feb 20, 2025 - 10:11 EST

Update

We are continuing to monitor for any further issues.
Posted Feb 19, 2025 - 19:28 EST

Monitoring

Delivery of notifications have resumed normal operations. Our team is continuing the investigation and conducting additional testing.
Posted Feb 19, 2025 - 19:08 EST

Investigating

We are currently investigating reports of Hypercare message notifications experiencing delays in delivery. Our team is actively investigating the issue. This is only affecting users in Canada.
Posted Feb 19, 2025 - 19:01 EST
This incident affected: Canadian Region (Notifications and Real-Time Syncing (Canadian Region)).