All the data has been recovered and the system is back to normal.
Posted Nov 20, 2020 - 01:24 EST
Update
Affected carriers: Recovered Order Sync: Recovered Webhook notification: Recovering (45 mins delay now)
Posted Nov 20, 2020 - 00:52 EST
Monitoring
Webhooks' maximum delay is 75 mins now, however, the system has started to recover and all the data should be restored soon. All affected couriers have already been restored.
Posted Nov 20, 2020 - 00:44 EST
Update
The impact of AWS issue is much more than what we had anticipated earlier: - Sending webhook notifications delayed by 50+ mins now - Delayed updates for around 20 carriers - Order sync will be delayed, however, no concern related to the data loss
The team is finding RTO methods to resume the services ASAP.
Posted Nov 20, 2020 - 00:37 EST
Identified
We are currently experiencing a 30 mins delay in webhooks processing due to an ongoing issue with our service provider (AWS). Hoping to get this resolved soon.
Posted Nov 19, 2020 - 23:54 EST
This incident affected: Secure Web Portal and Couriers Tracking System.