Long pending transactions - Custody API

Incident Report for Tangany

Resolved

The issue has been resolved. We apologize for any inconvenience. Kindly find the below summary.

Incident summary: Long pending transactions - awaiting confirmation

Issue start time:

Issue resolution time:

Severity: 2 PARTIAL OUTAGE

Impacted Services:
Eth mainnet, Genosis, Polygon Mumbai, Goerli,

Root cause: Long processing time made the pileup on external networks, System has been switched on our DR infrastructure, and re-processed all pending transactions.

Recovery: As per the quick response, the decision has been made to switch on the DR. Whereas, our team is on it to upgrade the infrastructure features to tackle these scenarios in the future.
Posted Mar 21, 2023 - 13:42 CET

Monitoring

Transactions are working as expected.
All pending transactions are being processed.
Posted Mar 15, 2023 - 18:40 CET

Update

We are continuing to work on a fix for this issue.
Posted Mar 15, 2023 - 17:26 CET

Identified

The issue has been identified and solution has been implemented.
Posted Mar 15, 2023 - 17:25 CET

Update

We are continuing to investigate this issue.
Posted Mar 15, 2023 - 11:22 CET

Investigating

Since 14.03.2023 03:49:00 CET transactions on supported protocols are pending with 'null' output.
We are currently investigating the root cause, and we will provide you timely updates about the incident.

Custody API response:
{
"process": "Pending",
"status": null,
"created": "2023-03-15T08:57:29Z",
"updated": "2023-03-15T08:57:29Z",
"output": null
}

Thank you for your patience.
Posted Mar 15, 2023 - 11:22 CET