BLiP Status

Track platform BLiP status


...
...
All Systems Operational
Uptime over the past 90 days. View historical uptime.
Desk Operational
90 days ago
99.69 % uptime
Today
BLiP Platform Operational
90 days ago
99.94 % uptime
Today
CRM ? Operational
90 days ago
99.97 % uptime
Today
Core Operational
90 days ago
99.85 % uptime
Today
Analytics Operational
90 days ago
99.94 % uptime
Today
Artificial Intelligence Operational
90 days ago
99.99 % uptime
Today
Portal Operational
90 days ago
99.89 % uptime
Today
Cloud Infrastructure Operational
90 days ago
99.98 % uptime
Today
Channels Operational
90 days ago
99.99 % uptime
Today
WhatsApp ? Operational
90 days ago
99.99 % uptime
Today
Telegram Operational
90 days ago
99.99 % uptime
Today
Messenger Operational
90 days ago
99.97 % uptime
Today
BlipChat Operational
90 days ago
100.0 % uptime
Today
Workplace Chat Operational
BusinessChat Operational
Skype Operational
E-mail Operational
Hosting Enterprise Operational
90 days ago
99.93 % uptime
Today
Bot Builder Operational
90 days ago
99.93 % uptime
Today
Bot Router Operational
90 days ago
99.93 % uptime
Today
Hosting Business Operational
90 days ago
99.77 % uptime
Today
Bot Builder Operational
90 days ago
99.59 % uptime
Today
Bot Router Operational
90 days ago
99.95 % uptime
Today
Hosting Standard Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
had a major outage
had a partial outage

No incidents or maintenance related to this downtime.

Past Incidents
Apr 3, 2020

No incidents reported today.

Apr 2, 2020

No incidents reported.

Apr 1, 2020
Desk Degradation
Resolved - It was identified that there was a slowness due to a blocking of Locks for the distribution of Tickets on the Desk.
We have an internal problem call to identify the root cause of this scenario, as soon as we have a case status we will report on that incident with the Post-Mortem.

Start of instability: 03/31/2020 at 3:53 pm
End of Instability: 01/04/2020 at 11:34
Apr 1, 12:43 GMT-03:00
Monitoring - We made new interventions on one of the desk processors and we are monitoring the scenario to validate if the problem will occur again
Apr 1, 12:08 GMT-03:00
Investigating - Dears, good morning!

We have identified a slowness for some DESK commands and we are already checking with priority what may be causing this scenario.
Apr 1, 10:06 GMT-03:00
Mar 31, 2020
Desk Degradation
Resolved - Impacts: Instability was identified in the service tool: Desk in the automatic distribution part and in the function to pull the ticket manually;

Actions: A problem ticket was created internally to map the root cause; #67056

Duration of instability:

Start: 16:00
End: 17:00
Mar 31, 20:07 GMT-03:00
Monitoring - Dear, good afternoon!

Our technical team has already made the necessary intervention in the ticket distribution / withdrawal service to correct the scenario and we are monitoring to ensure that the error no longer occurs.
Mar 31, 17:40 GMT-03:00
Identified - Identificamos uma degradação no Desk para puxar os Tickets.
Mar 31, 16:46 GMT-03:00
Mar 30, 2020

No incidents reported.

Mar 29, 2020

No incidents reported.

Mar 28, 2020

No incidents reported.

Mar 27, 2020
Desk e Portal.blip.ai unavailable
Postmortem - Read details
Mar 27, 22:04 GMT-03:00
Resolved - The environment is normalized after technical intervention. We have an internal problem to analyze the root cause.
Mar 27, 18:52 GMT-03:00
Monitoring - The problem has been identified and a fix is being implemented.
Mar 27, 18:39 GMT-03:00
Investigating - We are currently investigating this issue.
Mar 27, 18:25 GMT-03:00
Messenger channel degradation
Resolved - We verified normalization of failures that occurred due to changes made by Facebook.

We found that for normalization of the bot it is necessary to reconnect the bot on the Facebook page. Customers with problems are required to take this action.

We registered a ticket with Facebook to analyze the root cause of the failure.
Mar 27, 12:38 GMT-03:00
Update - We are in contact with Facebook to understand the changes. We will update as soon as we have new information.
Mar 27, 11:22 GMT-03:00
Investigating - We identified that there were changes in Facebook in relation to the Messenger channel, causing unavailability in some bots that use this channel.

We are working to understand the changes and adapt ourselves so that the restoration of these bots occurs as soon as possible.
Mar 27, 10:57 GMT-03:00
Degradation Http
Resolved - This incident has been resolved.
Mar 27, 05:30 GMT-03:00
Investigating - We are experiencing a degradation in our core for HTTP requests, so some http requests and commands can failed
Mar 27, 04:07 GMT-03:00
Mar 26, 2020

No incidents reported.

Mar 25, 2020
Desk e Portal.blip.ai unavailable
Resolved - The environment is normalized after technical intervention. We recorded a problem to analyze the root cause.
Mar 25, 16:11 GMT-03:00
Identified - The issue has been identified and a fix is being implemented.
Mar 25, 16:10 GMT-03:00
Update - We are continuing to investigate this issue.
Mar 25, 16:04 GMT-03:00
Investigating - We are currently investigating this issue.
Mar 25, 16:02 GMT-03:00
Mar 24, 2020
Degradation BLiP
Postmortem - Read details
Apr 1, 11:08 GMT-03:00
Resolved - Problem:

Slowness on the platform with impact on message interaction. This slowness occurred as a result of a 50% increase in the flow of traffic on the platform compared to the last 10 days.


Actions taken:

Throughout yesterday, 03/23, we implemented some maneuvers to try to mitigate the slowness on the platform, with an impact mainly on the interaction of messages.

All the actions taken were to avoid ScaleUp during the operation, because due to the high volume of transactions carried out at the Bank in question, the procedure could cause an unavailability with a greater impact on the client's business.


The ScaleUp of the database started at 9:00 pm and ended at 1:30 am.

We monitored throughout the morning and had no scenarios related to the problem.
Mar 24, 15:29 GMT-03:00
Monitoring - The ScaleUp of the database was processed during the night.
Processing was completed around 1:30 am.
We continue to monitor the platform.
Mar 24, 06:58 GMT-03:00
Investigating - Problem: Slowness on the platform with an impact on message interaction. This slowness was due to a significant increase in traffic flow compared to the last 10 days.

Actions taken: Throughout the day we implemented some maneuvers to try to mitigate the slowness on the platform with an impact on the interaction of messages.
All the actions taken were to avoid ScaleUp during the operation because due to the high volume of transactions carried out at the database in question, the procedure could cause unavailability with a greater impact on the client's business.

Actions in progress: ScaleUp in the database
Start date/time: It will start at 9 pm today (03/23/2020)
End forecast: estimated duration of 13 hours, this time can be minimized by the volume of traffic during the night.
Mar 23, 20:43 GMT-03:00
Update - The technical team is applying a configuration to some servers in order to resolve the slow points.

The forecast is at 5PM.
Mar 23, 16:33 GMT-03:00
Update - We are continuing to investigate this issue.

Some actions have already been taken to resolve the problem as mentioned in the last update.
Mar 23, 14:52 GMT-03:00
Update - The technical team continues to investigate the case, some maneuvers have already been carried out to solve the problem as mentioned in the last update. We saw an improvement, but we still have slow points in the exchange of messages.
Mar 23, 14:49 GMT-03:00
Identified - The impact is occurring on the Enterprise cluster.

Actions:

A new server was added in production. We are monitoring the performance of message processing.

Scale up was performed on the Transactional Database from 16vCores to 24vCores.
Mar 23, 12:39 GMT-03:00
Update - We are continuing to investigate this issue.
Mar 23, 11:49 GMT-03:00
Investigating - We identified slowness on the BLiP platform. the problem is being analyzed with priority and we will update as soon as we have new status.
Mar 23, 11:48 GMT-03:00
Mar 23, 2020
Manutenção Programada em caráter Emergencial / Emergency scheduled maintenance
Completed - The scheduled maintenance has been completed.
Mar 23, 05:00 GMT-03:00
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 23, 04:01 GMT-03:00
Scheduled - Data Inicial: Segunda 23-03-2020 04:00 (GMT-3)
Data Final: Segunda 23-03-2020 05:00 (GMT-3)
Duração: 1 hora
Tipo de manutenção: Manutenção Programada em caráter Emergencial
Severidade: Alta
Impacto: Indisponibilidade total

A equipe da Take irá realizar uma manutenção na infraestrutura durante a próxima madrugada, para ampliar a capacidade de processamento da plataforma. Durante a janela de manutenção serão esperadas instabilidades e até interrupções totais no serviço, que serão concentradas no menor tempo possível. Estamos com isso garantindo a infraestrutura necessária para suportar o crescimento exponencial no número de mensagens esperado para os próximos dias.

-----

Start date: Monday 23-03-2020 04:00 (GMT-3)
End date: Monday 23-03-2020 05:00 (GMT-3)
Duration: 1 hour
Type: Emergency scheduled maintenance
Severity: High
Impact: Total unavailability

The Take team will carry out a maintenance on the infrastructure during the next morning, to expand the processing capacity of the platform. During the maintenance window, instabilities and even total service interruptions will be expected, which will be concentrated in the shortest possible time. We are thus guaranteeing the necessary infrastructure to support the exponential growth in the number of messages expected in the coming days.
Mar 22, 11:49 GMT-03:00
Mar 22, 2020

No incidents reported.

Mar 21, 2020

No incidents reported.

Mar 20, 2020

No incidents reported.

← Incident History Powered by Statuspage

BLiP Areas

Help DeskBuilderChannelsIntegrationsArtificial intelligenceMetrics and AnalyticsRetention and Engagement

Resources

API ReferenceFaqGithub

Courses

Free CourseBotcamp

Community

BLiP BetaBLiP IdeasEventsFórum

Policies

Cookie PoliciesPrivacy PoliciesTerms of use

Copyright © 2020 BLiP Powered by Take