Event tracking not reflecting on personalization

All pending events have been processed and personalization is back to reflecting real-time data.

• We have rolled out a fix for the remaining issues and tracked events are now being processed.
• No data loss has been detected — events stored during the incident are being processed as part of catch-up.
• Estimated catch-up time: approximately 2–3 hours for all applications to process stored events and return to real-time processing.
• We are monitoring progress closely and will post a final update once catch-up is complete or sooner if anything changes.

Thank you for your patience.

• We have completed and validated a fix for the secondary issue that was triggered after the earlier deployment.
• We are preparing a rollout that includes this fix and will monitor the system closely during deployment.

We will provide further updates to this ticket as the rollout progresses and as we verify full recovery.

• The fix rolled out earlier was insufficient to fully restore processing.
• A separate, but related, issue was triggered following the deployment.
• Our engineering teams are actively investigating this new problem and working on containment and a resolution.

We continue to monitor system behavior closely and will provide further updates to this ticket as soon as we have more information. Thank you for your patience.

• We are rolling out a fix for the (de)serialization issue identified earlier.
• The rollout is in progress and we are actively monitoring the system for recovery and any regressions.

We will provide further updates to this ticket as the rollout completes and we verify normal processing has resumed. Thank you for your patience.

Our team traced the issue to one of our recent deployments and attempted a rollback of the affected applications.
The deployed changes were backwards compatible but not forward compatible, so the rollback did not restore normal processing.

• Root cause identified: changes to event schemas over recent deployments caused (de)serialization failures for some events.
• Next steps: we are developing and validating a fix to handle the updated schema correctly and will deploy it as soon as testing is complete.

We will post further updates to this ticket as progress is made. Thank you for your patience.

At 22:12 UTC we detected that newly tracked events on Croct were not being reflected in Croct’s personalization services (evaluations and slot contents). Customers may see delayed or stale personalization while we investigate.

Impact

• Affected: Personalization evaluation and slot content updates for newly tracked events.
• Not affected: Event ingestion — all incoming events are being accepted and stored.
• Data loss: We have detected no data loss. All events are being correctly stored and will be processed once the issue is resolved.

What we are doing right now

• Our engineering team is actively investigating the root cause and working to restore normal processing.
• We are validating the ingestion and processing pipelines, preparing and testing fixes, and monitoring system health closely.
• We will continue to prioritize recovery and correctness over speed to ensure no events are lost or misprocessed.

Customer guidance

• No action is required on your side. Events you send now are being stored and will be applied to personalization once processing is restored.
• If you depend on real-time personalization for critical flows, consider temporarily using cached defaults or fallbacks until processing resumes.
• If you see any behavior that you believe is unrelated to this incident, or you need urgent assistance, please contact via your account representative.

Updates
We will continue to report back on this ticket until the issue is resolved.

We apologize for the disruption and appreciate your patience while we resolve this.

Detected newly tracked events not affecting personalization.