Back to overview
Downtime

Marketing Website, Processing Server (Full + Snippet 1), and 21 other services are down

Jan 22, 2025 at 6:37am UTC
Affected services
Marketing Website
Processing Server (Full + Snippet 1)
Processing Server (Full + Snippet 2)
Processing Server (Full + Snippet 3)
Processing Server (Snippets + Full 4)
Processing Server (Snippets + Full 6)
Processing Server (Snippets + Full 7)
Processing Server (TikTok 0)
Processing Server (TikTok 1)
Processing Server (TikTok 2)
Processing Server (TikTok 3)
Processing Server (TikTok 4)
Processing Server (TikTok 5)
Processing Server (TikTok 6)
Processing Server (TikTok 7)
PROD Full /Snippet Server 0 - JobPriority Not 0
PROD Full /Snippet Server 1 - JobPriority Not 0
PROD Scheduler Server - JobPriority Not 0
TikTok 0 - JobPriority Not 0
TikTok 1 - JobPriority Not 0
TikTok 2 - JobPriority Not 0
TikTok 3 - JobPriority Not 0
Manual Update Cron (Source)

Resolved
Jan 24, 2025 at 11:34pm UTC

Marketing Website recovered.

Updated
Jan 24, 2025 at 11:28pm UTC

Marketing Website went down.

Updated
Jan 24, 2025 at 7:25pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 7:24pm UTC

PROD Full /Snippet Server 1 - JobPriority Not 0 and PROD Scheduler Server - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 7:23pm UTC

Processing Server (Full + Snippet 2) and Processing Server (Full + Snippet 3) recovered.

Updated
Jan 24, 2025 at 7:23pm UTC

Processing Server (Full + Snippet 1) recovered.

Updated
Jan 24, 2025 at 7:23pm UTC

Processing Server (Snippets + Full 6) and Processing Server (Snippets + Full 7) recovered.

Updated
Jan 24, 2025 at 7:19pm UTC

TikTok 3 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 7:19pm UTC

TikTok 2 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 7:19pm UTC

TikTok 0 - JobPriority Not 0 and TikTok 1 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 7:15pm UTC

Processing Server (Full + Snippet 3) went down.

Updated
Jan 24, 2025 at 7:15pm UTC

Processing Server (Full + Snippet 1), Processing Server (Full + Snippet 2), and Processing Server (Snippets + Full 7) went down.

Updated
Jan 24, 2025 at 7:15pm UTC

Processing Server (Snippets + Full 6) went down.

Updated
Jan 24, 2025 at 7:02pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 7:01pm UTC

PROD Full /Snippet Server 1 - JobPriority Not 0 and PROD Scheduler Server - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 6:56pm UTC

TikTok 1 - JobPriority Not 0 and TikTok 2 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 6:55pm UTC

TikTok 0 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 6:48pm UTC

Processing Server (TikTok 6) and Processing Server (TikTok 7) recovered.

Updated
Jan 24, 2025 at 6:40pm UTC

Processing Server (TikTok 6) and Processing Server (TikTok 7) went down.

Updated
Jan 24, 2025 at 6:40pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 6:39pm UTC

PROD Full /Snippet Server 1 - JobPriority Not 0 and PROD Scheduler Server - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 6:34pm UTC

TikTok 2 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 6:33pm UTC

TikTok 0 - JobPriority Not 0 and TikTok 1 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 6:17pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 6:17pm UTC

PROD Full /Snippet Server 1 - JobPriority Not 0 and PROD Scheduler Server - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 6:11pm UTC

TikTok 2 - JobPriority Not 0 and TikTok 3 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 6:11pm UTC

TikTok 0 - JobPriority Not 0 and TikTok 1 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 5:31pm UTC

Manual Update Cron (Source) recovered.

Updated
Jan 24, 2025 at 4:25pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 recovered.

Updated
Jan 24, 2025 at 4:22pm UTC

Processing Server (Snippets + Full 4) recovered.

Updated
Jan 24, 2025 at 4:17pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 went down.

Updated
Jan 24, 2025 at 4:14pm UTC

Processing Server (Snippets + Full 4) went down.

Updated
Jan 24, 2025 at 11:57am UTC

Manual Update Cron (Source) went down.

Updated
Jan 24, 2025 at 10:22am UTC

Processing Server (Snippets + Full 6) and Processing Server (Snippets + Full 7) recovered.

Updated
Jan 24, 2025 at 10:14am UTC

Processing Server (Snippets + Full 6) and Processing Server (Snippets + Full 7) went down.

Updated
Jan 24, 2025 at 12:02am UTC

Processing Server (TikTok 4) recovered.

Updated
Jan 24, 2025 at 12:01am UTC

Processing Server (TikTok 2) recovered.

Updated
Jan 23, 2025 at 11:47pm UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 23, 2025 at 11:41pm UTC

Processing Server (TikTok 6) went down.

Updated
Jan 23, 2025 at 11:41pm UTC

Processing Server (TikTok 4) went down.

Updated
Jan 23, 2025 at 11:40pm UTC

Processing Server (TikTok 2) went down.

Updated
Jan 23, 2025 at 11:16pm UTC

Processing Server (TikTok 2) and Processing Server (TikTok 4) recovered.

Updated
Jan 23, 2025 at 11:10pm UTC

Processing Server (TikTok 4) went down.

Updated
Jan 23, 2025 at 11:10pm UTC

Processing Server (TikTok 2) went down.

Updated
Jan 23, 2025 at 3:32pm UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 23, 2025 at 3:32pm UTC

Processing Server (TikTok 5) recovered.

Updated
Jan 23, 2025 at 3:31pm UTC

Processing Server (TikTok 4) recovered.

Updated
Jan 23, 2025 at 3:31pm UTC

Processing Server (TikTok 2) and Processing Server (TikTok 3) recovered.

Updated
Jan 23, 2025 at 3:30pm UTC

Processing Server (TikTok 1) recovered.

Updated
Jan 23, 2025 at 3:24pm UTC

Processing Server (TikTok 6) went down.

Updated
Jan 23, 2025 at 3:24pm UTC

Processing Server (TikTok 5) went down.

Updated
Jan 23, 2025 at 3:24pm UTC

Processing Server (TikTok 4) went down and PROD Full /Snippet Server 0 - JobPriority Not 0 recovered.

Updated
Jan 23, 2025 at 3:23pm UTC

Processing Server (TikTok 3) went down and PROD Scheduler Server - JobPriority Not 0 recovered.

Updated
Jan 23, 2025 at 3:23pm UTC

Processing Server (TikTok 2) went down and PROD Full /Snippet Server 1 - JobPriority Not 0 recovered.

Updated
Jan 23, 2025 at 3:22pm UTC

Processing Server (TikTok 1) went down.

Updated
Jan 23, 2025 at 3:18pm UTC

TikTok 3 - JobPriority Not 0 recovered.

Updated
Jan 23, 2025 at 3:18pm UTC

TikTok 1 - JobPriority Not 0 and TikTok 2 - JobPriority Not 0 recovered.

Updated
Jan 23, 2025 at 3:17pm UTC

TikTok 0 - JobPriority Not 0 recovered.

Updated
Jan 23, 2025 at 3:01pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 went down.

Updated
Jan 23, 2025 at 3:00pm UTC

PROD Full /Snippet Server 1 - JobPriority Not 0 and PROD Scheduler Server - JobPriority Not 0 went down.

Updated
Jan 23, 2025 at 2:55pm UTC

TikTok 3 - JobPriority Not 0 went down.

Updated
Jan 23, 2025 at 2:55pm UTC

TikTok 1 - JobPriority Not 0 and TikTok 2 - JobPriority Not 0 went down.

Updated
Jan 23, 2025 at 2:54pm UTC

TikTok 0 - JobPriority Not 0 went down.

Updated
Jan 23, 2025 at 12:03pm UTC

Marketing Website recovered.

Updated
Jan 23, 2025 at 11:57am UTC

Marketing Website went down.

Updated
Jan 23, 2025 at 10:25am UTC

Manual Update Cron (Source) recovered.

Updated
Jan 23, 2025 at 1:57am UTC

Processing Server (TikTok 0) recovered.

Updated
Jan 23, 2025 at 1:51am UTC

Processing Server (TikTok 0) went down.

Updated
Jan 23, 2025 at 1:46am UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 23, 2025 at 1:39am UTC

Processing Server (TikTok 6) went down.

Updated
Jan 23, 2025 at 1:15am UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 23, 2025 at 1:09am UTC

Processing Server (TikTok 6) went down.

Updated
Jan 23, 2025 at 12:45am UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 23, 2025 at 12:39am UTC

Processing Server (TikTok 6) went down.

Updated
Jan 22, 2025 at 5:31pm UTC

TikTok 1 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:31pm UTC

TikTok 0 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:30pm UTC

Processing Server (TikTok 6) and Processing Server (TikTok 7) recovered.

Updated
Jan 22, 2025 at 5:30pm UTC

Processing Server (TikTok 5) recovered.

Updated
Jan 22, 2025 at 5:29pm UTC

Processing Server (TikTok 2), Processing Server (TikTok 3), and Processing Server (TikTok 4) recovered.

Updated
Jan 22, 2025 at 5:23pm UTC

Processing Server (TikTok 6) and Processing Server (TikTok 7) went down.

Updated
Jan 22, 2025 at 5:22pm UTC

Processing Server (TikTok 5) went down and PROD Full /Snippet Server 0 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:22pm UTC

PROD Scheduler Server - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:21pm UTC

Processing Server (TikTok 2), Processing Server (TikTok 3), and Processing Server (TikTok 4) went down and PROD Full /Snippet Server 1 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:16pm UTC

TikTok 3 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:16pm UTC

TikTok 2 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 5:14pm UTC

PROD Full /Snippet Server 0 - JobPriority Not 0, PROD Full /Snippet Server 1 - JobPriority Not 0, and PROD Scheduler Server - JobPriority Not 0 went down.

Updated
Jan 22, 2025 at 5:08pm UTC

TikTok 3 - JobPriority Not 0 went down.

Updated
Jan 22, 2025 at 5:08pm UTC

TikTok 2 - JobPriority Not 0 went down.

Updated
Jan 22, 2025 at 5:08pm UTC

TikTok 0 - JobPriority Not 0 and TikTok 1 - JobPriority Not 0 went down.

Updated
Jan 22, 2025 at 4:27pm UTC

Processing Server (TikTok 0) recovered.

Updated
Jan 22, 2025 at 4:21pm UTC

Processing Server (TikTok 0) went down.

Updated
Jan 22, 2025 at 4:15pm UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 22, 2025 at 4:09pm UTC

Processing Server (TikTok 6) went down.

Updated
Jan 22, 2025 at 4:04pm UTC

Manual Update Cron (Source) went down.

Updated
Jan 22, 2025 at 3:57pm UTC

Processing Server (TikTok 0) recovered.

Updated
Jan 22, 2025 at 3:51pm UTC

Processing Server (TikTok 0) went down.

Updated
Jan 22, 2025 at 3:45pm UTC

Processing Server (TikTok 6) recovered.

Updated
Jan 22, 2025 at 3:39pm UTC

Processing Server (TikTok 6) went down.

Updated
Jan 22, 2025 at 7:36am UTC

PROD Full /Snippet Server 0 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 7:36am UTC

PROD Full /Snippet Server 1 - JobPriority Not 0 and PROD Scheduler Server - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 7:35am UTC

Processing Server (Full + Snippet 1), Processing Server (Full + Snippet 2), and Processing Server (Full + Snippet 3) recovered.

Updated
Jan 22, 2025 at 7:31am UTC

TikTok 3 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 7:30am UTC

TikTok 2 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 7:30am UTC

TikTok 0 - JobPriority Not 0 and TikTok 1 - JobPriority Not 0 recovered.

Updated
Jan 22, 2025 at 7:27am UTC

Processing Server (Full + Snippet 2) and Processing Server (Full + Snippet 3) went down.

Updated
Jan 22, 2025 at 7:26am UTC

Processing Server (Full + Snippet 1) went down.

Updated
Jan 22, 2025 at 6:58am UTC

PROD Full /Snippet Server 0 - JobPriority Not 0, PROD Full /Snippet Server 1 - JobPriority Not 0, and PROD Scheduler Server - JobPriority Not 0 went down.

Updated
Jan 22, 2025 at 6:38am UTC

TikTok 3 - JobPriority Not 0 went down.

Updated
Jan 22, 2025 at 6:37am UTC

TikTok 1 - JobPriority Not 0 and TikTok 2 - JobPriority Not 0 went down.

Created
Jan 22, 2025 at 6:37am UTC

TikTok 0 - JobPriority Not 0 went down.