The issue has been resolved. We apologize for any inconvenience. Kindly find the summary below. Incident summary: All wallets that have been migrated formerly from v1 to v2 and for those partners whose data were involved in cleansing at v1 site were impacted and their newly migrated wallets were not usable anymore. Impacted Services: Custody(Migrated wallets) Root cause: All wallet deletions executed by the maintenance team were, therefore, unexpectedly synced to v2- as confirmed by the call history of the migrate-sync function. All keys of migrated wallets received the “disabled” property in the partner’s respective v2 KeyVaults. All newly created wallets in v2 are operational and not touched. Impact start time: 21 Dec 2023, 14:40 CEST Impact End time: 29 Dec 2023, 16:40 CEST
Posted Jan 22, 2024 - 17:17 CET
Monitoring
The issue has been fixed, and we are monitoring the service custody for the migrated wallets. Whereas there is no data loss and impacted wallets have been re-activated again.
As of now, wallets have been restored. We will keep monitoring till the assurance of successful validation of all wallets.
Activity start time: 21 Dec 2023, 14:40 CEST
Activity End time: 29 Dec 2023, 16:40 CEST
Monitoring duration: 6 hours
Posted Dec 29, 2023 - 18:42 CET
Identified
We have identified the problem and are restoring the migrated custody wallets. We apologize for any inconvenience.
Activity start time: 21 Dec 2023, 14:40 CEST
Activity End time: 29 Dec 2023, 16:40 CEST
Posted Dec 29, 2023 - 17:45 CET
Investigating
We are currently experiencing a partial outage of our Custody v1 migrated wallets on the v2 solution. Whereas there is no data loss and the solution is just a 're-enable' execution again for impacted wallets. Our team is currently working to restore them as soon as possible. We apologize for any inconvenience. Impacted Service: Custody (v1 migrated wallets on v2) Activity start time: 21 Dec 2023, 14:40 CEST
Posted Dec 29, 2023 - 16:13 CET
This incident affected: Tangany Custody API (API backend).