Degraded performance on the BLiP
Incident Report for Blip
Postmortem

Hi blipper! On 10/08/2021, Friday, we faced an unavailability on Blip that affected the operation of many Smart Contacts.

To be transparent with you, Blip user, we are writing to tell you what happened.

What happened

A failure in connection with one of our databases was identified, which is used to store conversation history, causing instabilities in the conversation flows of the Blip Desk functionality.

How this issue impact you

Because of this failure, the Blip Desk, our human service functionality, faced problems in executing message exchanges.

In the name of Take Blip, I want to say sorry for any problems caused to you, your company and your customers.

What we do to solve it

As soon as we identified the problem at 13:04pm -3UTC, our team put together and acted quickly to start the treatments. The correction was immediately applied and the service was normalized at 19:18pm -3UTC.

Where we are now

The Blip Desk is working again, and our technical team is following up with the cloud provider to identify the main cause of the failure.  In addition, we have internal actions to prevent events like that again.

You can check this history and all other Blip features status on our Status Page.

We also want to say thank you for your patience and remind you that we are always here to help you in any need. Just open a request on our Support or create a new topic on Blip Forum, the exclusive space to the whole users' community.

Sincerely,

Posted Oct 15, 2021 - 19:23 GMT-03:00

Resolved
Fault identified:

During our monitoring, we identified a failure in the message history of tickets in the Desk.

Impact for customers:

We faced a database server failure that caused inconsistencies in the display of message histories in Desk and caused problems logging into the platform.


Solution:

Our team performed the intervention by sizing the database solving the problem.

We are studying improvements to prevent this from happening again.

More information about the root cause will be made available on postmortem.

Start date/time: 08/10/2021 13:04
End date/time: 08/10/2021 19:18
Posted Oct 08, 2021 - 21:11 GMT-03:00
Update
We are still working on a definitive fix.
Posted Oct 08, 2021 - 16:40 GMT-03:00
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Oct 08, 2021 - 14:14 GMT-03:00
Investigating
We are suffering a degradation in the performance of the BLiP platform, our technical team is already working on the case.
Posted Oct 08, 2021 - 13:15 GMT-03:00
This incident affected: Desk.