Quality of Service Disruption Alert

Incident Report for Maropost Marketing Cloud

Postmortem

On Friday, Sept 30 at 4:24PM Eastern, our internal monitoring systems alerted us that the root domain, maropost.com, was non-responsive.

An investigation was immediately initiated. We discovered the root cause was that our cloud hosting provider had made a sudden and unannounced configuration change which caused wide-spread disruption to our platform.

Service restoration began 30 minutes later, and service fully restored and verified within 60 minutes.

Please consult with your Customer Success Manager for details regarding any impact to your account.

We are initiating discussions with our cloud hosting provider to ensure that this type of unannounced change does not reoccur.

Posted Oct 04, 2022 - 13:06 EDT

Resolved

Our Engineering Team has confirmed that system health has been fully restored to baseline norms.

All systems should now be fully operational for all clients. Thank you for your patience as we worked through this.

A Post Mortem of this incident will be posted within 2 business days.

If you continue to experience issues, please submit a ticket with our Support Team via the support page on Maropost Galaxy https://galaxy.maropost.com/s/contactsupport

Posted Sep 30, 2022 - 19:49 EDT

Monitoring

Our Engineering Team has successfully implemented the solution to restore system availability. We are continuing to monitor system health to ensure that all aspects of our platform are fully recovered.

We apologize for the inconvenience, and will post an update as soon as we've confirmed the issue is fully resolved

Posted Sep 30, 2022 - 17:42 EDT

Identified

Subsequent investigation has revealed that the incident reported at 4:29PM Eastern Time Zone on Friday, Sept. 30, 2022 was caused by a service interruption from one of our providers.

Our Engineering Team is monitoring the progress with our provider to ensure that service is restored as quickly as possible. In the meantime, we are taking steps ourselves to mitigate our platform’s exposure to this incident, and restore system availability to our clients.

At this point, we have ascertained the following components are affected:
- Application Login
- Campaign sends
- Tracking campaign responses including opens, clicks, bounces, unsubscribes, and complaints
- Web tracking
- Hosted Landing Pages
- Live Content
- REST API Service
- SMTP API Service
- FTP Server
- Journeys
- Data Journeys
- Mobile App Messaging

We apologize for the impact to your business. Thank you for your patience as we work to resolve this issue

Posted Sep 30, 2022 - 17:20 EDT

Investigating

At 4:29PM Eastern Time Zone on Friday, Sept. 30, 2022, we became aware of a service disruption affecting all clients.

Our Engineering Team is currently assessing the impact. We will post a status update as quickly as possible.

We apologize for the impact to your business. Thank you for your patience as we work to resolve this issue.

Posted Sep 30, 2022 - 17:04 EDT

This incident affected: Cloud Cluster (Application, API Server), App Cluster (Application, API Server), Cloud1 Cluster (Application, API Server), CA1 Cluster (Application, API Server), and EU1 Cluster (Application, API Server).