Almaz Murzabekov

I build data platforms that do not fit on one machine. More than ten years on high-load and big-data systems, pipelines at petabyte scale. These days that is GCP: Airflow and dbt for the pipelines, Terraform for the ground they stand on, and multi-agent systems on Vertex AI that take the tedious half of ingestion off my hands. With AI everywhere, the bottleneck moved back to the data, which is where I have always worked. I write down the parts that took longer to understand than they should have.

Stack

Data platforms

Warehouse and lakehouse
BigQuery, Databricks, Spark, Delta Lake
Pipelines
Airflow, dbt, Kafka, Structured Streaming
Infrastructure
Terraform, Cloud Run, GitHub Actions, Workload Identity
Agents
Google ADK, Vertex AI Agent Engine

Language models

Training
PyTorch, FSDP2, torch.compile, FlashAttention
Measurement
lm-evaluation-harness, trainmeter
Inference engines
vLLM, SGLang, llama.cpp, Ollama
Languages
Python, SQL, Rust

Projects

  • WIP

    almz

    A family of language models trained from zero, up to 1.04B parameters on 20B tokens.

  • WIP

    trainmeter

    Where the FLOPs of a training run go and how well the GPUs are used, with the training code unchanged.

  • PyTorch exercises for LLM developers

    Twenty drills on PyTorch for LLM work, from broadcasting to a transformer block, AdamW and a KV cache.

    Almaz-KG/pytorch-exercises-for-llm-developers

  • WIP

    PyTorch distributed exercises for LLM developers

    Thirty drills on distributed PyTorch, from collectives to FSDP, pipeline parallelism and disaggregated serving.

  • nookat

    A lightweight desktop app for managing containers, built with Rust, Tauri and React.

    nookat-io/nookat

  • db-academy.io

    Closed now: A learning path for engineers to build their own database from scratch in Rust, from SQL parser to MVCC.

    db-academy-io