Degradation Http
Incident Report for Blip
Postmortem

We inform that today (02/23/2021), at around 05:00 am in scheduled update we had an upload of a release in the http module on one of our servers in a cluster composed of 6 servers. An abnormal behavior was observed by the monitoring on the server that was updated in the Http module, however, due to the fact that it is an isolated scenario, it was difficult to relate the identified failure with the alteration made early in the morning. After some analyzes and evaluations together with the technical team responsible for the platform, it was found that the errors were due to the update mentioned above.

Impact to the customer:

Failed to exchange messages on bots using http requests;

Http requests using “https://msging.net”;

In the display of the number of tickets in the attendance tool for the attendant, as it is done via http request.

Correction Applied:

The update was rollback to the referred server.

Start date / time: 23/02/2021 05:13

End date / time: 2/23/2021 7:45 PM

Posted Feb 23, 2021 - 22:29 GMT-03:00

Resolved
We inform that today (02/23/2021), at around 05:00 am in scheduled update we had an upload of a release in the http module on one of our servers in a cluster composed of 6 servers.
An anomalous behavior was observed by the monitoring on the server that was updated in the Http module, however, due to the fact that it is an isolated scenario, it was difficult to relate the identified failure with the alteration made early in the morning. After some analyzes and evaluations together with the technical team responsible for the platform, it was found that the errors were due to the update mentioned above.

Impact to the customer:

Failed to exchange messages on bots using http requests;
Http requests using “https://msging.net”;
In the display of the number of tickets in the attendance tool for the attendant, as it is done via http request.


Correction Applied:

The update was rollback to the referred server.

Start date / time: 23/02/2021 05:13
End date / time: 2/23/2021 7:45 PM
Posted Feb 23, 2021 - 22:24 GMT-03:00
Identified
We are experiencing a degradation in our core for HTTP requests, so some http requests and commands can failed
Posted Feb 23, 2021 - 05:13 GMT-03:00
This incident affected: Blip Platform (Core, Analytics, Portal).