Blip Portal and Desk [Low performance]
Incident Report for Blip
Postmortem

Start Time: 14/11/2024 ~05:21 (GMT-3)
End Time: 14/11/2024 ~11:35 (GMT-3)

Incident Summary:
The Desk and Portal are inaccessible for some customers.

Impact Analysis:
In short, the Portal presents a completely gray screen for some users and the Desk displays the Blip icon loading page (which keeps loading indefinitely).

Mitigation Measures:
We applied a workaround to our network infrastructure to mitigate the issue for all customers and received positive feedback from them that the platform is back to normal.

Root Cause Analysis:
In the face of the normalization of the environment, we classify the incident as resolved and now enter the root cause analysis stage.

This process will be conducted internally, as quickly as the situation requires, and will be published here on the status page as soon as possible.

If you have any questions or need additional clarification, we recommend that you contact the CSM of your account for support.

Posted Nov 14, 2024 - 12:11 GMT-03:00

Resolved
The workaround in our environment has restored the functionality and usability of our platform.
Posted Nov 14, 2024 - 12:08 GMT-03:00
Monitoring
Mitigation Measures:
We applied a workaround to our network infrastructure to mitigate the issue for all customers and received positive feedback from them that the platform is back to normal.

Given this feedback, we monitor the environment to make sure it remains stable.
Posted Nov 14, 2024 - 11:25 GMT-03:00
Identified
Incident Summary:
The Desk and Portal are inaccessible for some customers.

Impact Analysis:
In short, the Portal presents a completely gray screen for some users and the Desk displays the Blip icon loading page (which keeps loading indefinitely).

Current status:
Our technical experts are gathered in a war room investigating the scenario with full focus.

The analysis revealed that the failure was due to a failure in our supplier that controls and releases resources on the Blip platform.

We opened a ticket with them and requested priority service.

At the same time, we found a workaround in our network infrastructure to mitigate the problem and have already received positive feedback from some customers that this action resolved the problem for them. Given this validation, we are currently implementing the same solution for other customers.
Posted Nov 14, 2024 - 10:38 GMT-03:00
Update
Incident Summary:
The Desk and Portal are inaccessible for some customers.

Impact Analysis:
In short, the Portal presents a completely gray screen for some users and the Desk displays the Blip icon loading page (which keeps loading indefinitely).

Current status:
Our technical experts are gathered in a war room investigating the scenario with full focus.

We are validating the integrations that are part of the redirection process for user login.

As soon as we have new information, we will keep everyone updated.
Posted Nov 14, 2024 - 09:07 GMT-03:00
Update
We are continuing to investigate this issue.
Posted Nov 14, 2024 - 07:00 GMT-03:00
Investigating
We are currently experiencing low performance on Blip Portal.

Impact:
Slowness when using Blip Portal.

Update:
Our support team is actively working on it.
Posted Nov 14, 2024 - 06:45 GMT-03:00
This incident affected: Desk and Blip Platform (Portal).