CrazyGames data engineer test. Daniil Kovekh

Keep the events

What the brief asks, how large the traffic is, and the architecture I would ship. Demo boards at the end. Numbers marked brief, public, or estimate.

Task

Review the CrazyGames analytics pipeline and say how to improve it: monitoring, closer-to-real-time streaming, and company-wide LLM access with cost and permission control.

What I send

A written report and a small running demo of the same hops. Airflow every 20 minutes, Postgres as the warehouse stand-in, two Grafana boards. Synthetic events only.

What the brief forbids

A report that is mostly generated text with no decisions. AI as a daily tool is fine. The architecture choices and the reasons for them have to be mine.

Problem

About 4.000.000 daily players feed product, recommendations, developer stats, and ads through one event path. Five facts matter.

Data

Scale from the brief, their job posts, AWS, and company press.

4.000.000
DAU in the test brief. Working number for this review.
~10.000.000
Gameplays per day, from ~300.000.000 / month (job post)
50 to 60M
Monthly players, public 2025 to 2026
80 to 200M
Events per day, estimate at 20 to 50 per DAU
MetricValueSource
Daily active users4.000.000Test brief
Daily active users3.000.000+Product Analyst job
Monthly players50 to 60.000.000Cloudflare / company posts
Games on platform3.000+AWS Games Blog 2024
Gameplays / month~300.000.000Data Engineer job
Gameplays in 20263.000.000.000Company press Oct 2026
Playtime / day> 2.000.000 hProduct Analyst job
Playtime YTD to Sep 2026700.000.000 hCPO public post
Ingest average (estimate)~1.000 to 2.300 / sFrom event estimate
Ingest peak (estimate)~3.000 to 10.000 / sAbout 3 to 5× average
Per 20-min window (estimate)~1 to 3M rows72 windows / day

Event sources in the brief: portal, game frame (GF), SDK, ads. Server-side bot filter, integrity check, field validation, schema cleanup, enrichment.

Architecture today

As drawn in the brief. Batch warehouse, not a stream.

ClientsPortal, GF, SDK, Ads
Analytics ECSFilter, validate, enrich
Memory batchBy event type. Lossy on kill.
S3 JSONDurable files
RedshiftCOPY every 20 min
dbtHourly and daily
ReadersPreset, stats, model

Pink border marks the silent loss hop. Everything after S3 can retry a file that exists.

Architecture proposed

Keep the batch spine. Fix the lossy hop. Add one narrow fast path. Put Claude behind a query tool.

ClientsUnchanged
AnalyticsFlush JSON before forget
S3Replay source
Redshift + dbtRaw to marts
Claude toolSSO role, marts only

Narrow stream

The same S3 flush feeds a small consumer for game_start and game_end into a stats table (about one minute). The developer portal reads that. Hourly dbt stays the correction. Upsert by event_id.

What I would not do

Replace the whole 20-minute COPY with a full stream of every event type. At this volume that buys duplicates, out-of-order repairs, and a larger bill, for screens nobody is watching live.

Live demo

Same hops on our Airflow and Grafana. Synthetic traffic. Use it to show buffer loss, not production volume.

Trigger the DAG with {"crash_buffer": true} to drop one in-memory batch. Events lost in the buffer rises. Rows already loaded stay.