Partial outage and performance issues on Canadian Region

Incident Report for Hypercare

Resolved

This incident has been resolved.
Posted May 18, 2023 - 09:49 EDT

Update

Our test have shown that we are on the path to a more stable system. We will keep the status set to monitoring but as of now our system is operational.

Again, we apologize for the downtime that your organization experienced today.
Posted May 17, 2023 - 15:41 EDT

Update

The service has been restored. We are still monitoring the situation and attempting to identify the root cause of the issue. However, our team is running tests which could affect the performance of our platform.
Posted May 17, 2023 - 14:13 EDT

Update

The system is back up and fully operational now. We are still monitoring the situation to ensure it continues to maintain stability.
Posted May 17, 2023 - 12:55 EDT

Update

Some of our servers are back up but there’s a Hypercare service feature degradation. During a feature degradation, some features will be running at less than optimal performance. We’re investigating the issue and an update will be provided in 30 minutes or sooner if we have more information.
Posted May 17, 2023 - 11:21 EDT

Update

We are continuing to monitor for any further issues.
Posted May 17, 2023 - 10:42 EDT

Update

We are continuing to monitor for any further issues.
Posted May 17, 2023 - 10:33 EDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted May 17, 2023 - 10:32 EDT

Update

The Hypercare Technology team has noted a service disruption at our Canadian data centre. Our Canadian customers will experience issues access Hypercare on-call schedules, contact information and sending secure messages. The team is now exploring a path to resolution. We will provide an update in 30 minutes or sooner with additional information as it becomes available.
Posted May 17, 2023 - 10:14 EDT

Update

We are continuing to investigate this issue.
Posted May 17, 2023 - 10:12 EDT

Update

We are continuing to investigate this issue.
Posted May 17, 2023 - 09:39 EDT

Update

We are continuing to investigate this issue.
Posted May 17, 2023 - 09:25 EDT

Investigating

On May 17th, 2023 at 9:05 PM EST, we became aware of a service disruption on the Canadian region. We have engaged our response team and are working with urgency to resolve this incident as quick as possible. We apologize for the issues you may be facing due to this incident.

We will provide an update within the hour on the status of this incident.
Posted May 17, 2023 - 09:15 EDT
This incident affected: Canadian Region (Login and Single Sign On (Canadian Region), Messaging (Canadian Region), Notifications and Real-Time Syncing (Canadian Region), File Attachments (Canadian Region), Viewing Who is On-Call (Canadian Region), Code Teams (Canadian Region), Self-serve Scheduling (Canadian Region), Administration and Scheduling (Canadian Region)), App Store Availability (Google Play Store, Apple App Store), and Marketing Website (Global), Payment Portal.