We are looking for a
Senior Data Platform Engineer
to help drive the evolution of our
Azure-based Data Platform
and
Lakehouse architecture
.
This role is focused on the
platform itself
. The professional will own and evolve the
foundations
that data product teams build on:
Apache Airflow orchestration (migrating from ADF)
,
Azure resource management
(Key Vault, identity and access, networking, storage),
Azure DevOps CI/CD and Infrastructure as Code
,
Databricks and Unity Catalog governance
,
observability
and
cost efficiency
, while remaining
hands-on
and close to the teams that use the platform.
Key Responsibilities Orchestration Platform (Apache Airflow & ADF)
Own the
Apache Airflow
platform: environments, deployment, upgrades, configuration, connections, variables, pools and
DAG standards
.
Work on the migration from
Azure Data Factory to Airflow
and the evolution of the
Metadata-Driven Orchestration Framework
(DAG generation, reusable operators, dependency and scheduling patterns).
Ensure Airflow is
reliable and observable
: SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.
Azure Resource Management
Manage Azure resources with
Infrastructure as Code (Terraform)
: Databricks workspaces,
Data Lake Storage
, Data Factory,
Key Vault
, networking and supporting services.
Own
secrets and identity management
: Key Vault, service principals and managed identities, secret scopes and rotation.
Own
access management
: Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following
least privilege
, plus access requests and periodic reviews.
Own
Azure DevOps
: repositories,
CI/CD pipelines
, branch policies and
multi-environment promotion
, including deployment validation and Terraform state.
Track
cost and governance
of Azure resources: tagging, budgets, right-sizing and cost attribution (e.g. query tagging).
Hands-on Data Platform evolution
Contribute to the implementation of the
Data Catalog
and
Data Quality Framework
and it’s
lineage
across the platform.
Drive
observability
: pipelines that are observable, debuggable and alert-driven, with clear ownership of failures.
Operate the
Databricks
platform:
Unity Catalog
structure, cluster and job policies,
Auto Loader
ingestion patterns, and performance and cost optimization of workloads.
Technical Leadership
Participate in
architecture reviews
and technical decision-making, stating
trade-offs
(performance, cost, reliability, complexity) with data.
Define and promote
engineering standards
: naming conventions, patterns and guidelines, through code reviews and design reviews.
Collaborate with Architects, Product Owners and data engineers in
technical maturation
of data products.
Build and maintain
developer tooling and automation
, including
AI-assisted engineering
workflows (shared agent instructions, skills, automated PR review).
Support & Operations (Run Team Collaboration)
Provide
second-line support
to the Run/Operations team for complex production topics.
Troubleshoot and resolve
platform incidents
, failures and performance issues; lead
root cause analysis
and post-mortems.
Ensure proper
handover, documentation and operational readiness
of platform changes.
Continuously improve
runbooks, monitoring and operational processes
.
Senior Data Platform Engineer in Warsaw – pracodawca: Margo
Margo to doskonały pracodawca, który oferuje dynamiczne środowisko pracy w sercu Warszawy, sprzyjające innowacjom i współpracy. Nasza kultura pracy opiera się na zaufaniu i otwartości, co pozwala pracownikom na rozwój zawodowy oraz zdobywanie nowych umiejętności w obszarze testowania aplikacji bankowych. Oferujemy elastyczne godziny pracy oraz możliwość uczestnictwa w projektach z różnorodnymi partnerami biznesowymi, co czyni naszą ofertę wyjątkową i atrakcyjną dla ambitnych specjalistów.