Senior Data Platform Engineer in Warsaw

Senior Data Platform Engineer in Warsaw

Warsaw Stacjonarnie
M

We are looking for a

Senior Data Platform Engineer

to help drive the evolution of our

Azure-based Data Platform

and

Lakehouse architecture

.

This role is focused on the

platform itself

. The professional will own and evolve the

foundations

that data product teams build on:

Apache Airflow orchestration (migrating from ADF)

,

Azure resource management

(Key Vault, identity and access, networking, storage),

Azure DevOps CI/CD and Infrastructure as Code

,

Databricks and Unity Catalog governance

,

observability

and

cost efficiency

, while remaining

hands-on

and close to the teams that use the platform.

Key Responsibilities Orchestration Platform (Apache Airflow & ADF)

Own the

Apache Airflow

platform: environments, deployment, upgrades, configuration, connections, variables, pools and

DAG standards

.

Work on the migration from

Azure Data Factory to Airflow

and the evolution of the

Metadata-Driven Orchestration Framework

(DAG generation, reusable operators, dependency and scheduling patterns).

Ensure Airflow is

reliable and observable

: SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.

Azure Resource Management

Manage Azure resources with

Infrastructure as Code (Terraform)

: Databricks workspaces,

Data Lake Storage

, Data Factory,

Key Vault

, networking and supporting services.

Own

secrets and identity management

: Key Vault, service principals and managed identities, secret scopes and rotation.

Own

access management

: Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following

least privilege

, plus access requests and periodic reviews.

Own

Azure DevOps

: repositories,

CI/CD pipelines

, branch policies and

multi-environment promotion

, including deployment validation and Terraform state.

Track

cost and governance

of Azure resources: tagging, budgets, right-sizing and cost attribution (e.g. query tagging).

Hands-on Data Platform evolution

Contribute to the implementation of the

Data Catalog

and

Data Quality Framework

and it’s

lineage

across the platform.

Drive

observability

: pipelines that are observable, debuggable and alert-driven, with clear ownership of failures.

Operate the

Databricks

platform:

Unity Catalog

structure, cluster and job policies,

Auto Loader

ingestion patterns, and performance and cost optimization of workloads.

Technical Leadership

Participate in

architecture reviews

and technical decision-making, stating

trade-offs

(performance, cost, reliability, complexity) with data.

Define and promote

engineering standards

: naming conventions, patterns and guidelines, through code reviews and design reviews.

Collaborate with Architects, Product Owners and data engineers in

technical maturation

of data products.

Build and maintain

developer tooling and automation

, including

AI-assisted engineering

workflows (shared agent instructions, skills, automated PR review).

Support & Operations (Run Team Collaboration)

Provide

second-line support

to the Run/Operations team for complex production topics.

Troubleshoot and resolve

platform incidents

, failures and performance issues; lead

root cause analysis

and post-mortems.

Ensure proper

handover, documentation and operational readiness

of platform changes.

Continuously improve

runbooks, monitoring and operational processes

.

Senior Data Platform Engineer in Warsaw – pracodawca: Margo

Margo to doskonały pracodawca, który oferuje dynamiczne środowisko pracy w sercu Warszawy, sprzyjające innowacjom i współpracy. Nasza kultura pracy opiera się na zaufaniu i otwartości, co pozwala pracownikom na rozwój zawodowy oraz zdobywanie nowych umiejętności w obszarze testowania aplikacji bankowych. Oferujemy elastyczne godziny pracy oraz możliwość uczestnictwa w projektach z różnorodnymi partnerami biznesowymi, co czyni naszą ofertę wyjątkową i atrakcyjną dla ambitnych specjalistów.

M

Dane kontaktowe:

Zespół rekrutacyjny Margo