We are looking for a Senior Data Engineer to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture.
This role is focused on Data Products development. The successful candidate will build pipelines in ADF, DAGs in Airflow and work closely with POs to refine business requirements and transform business needs into reliable Data Products in Databricks.
Key Responsibilites Hands-on Data Product development
Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow.
Develop data transformations in Azure Databricks (PySpark / SQL) to create/extend data products following the Medallion Architecture layers: Bronze (raw ingestion)
Silver (cleansed & conformed)
Gold (business-ready data products)
Optimize performance, cost, and reliability of data workloads.
Raise important questions during business maturation in order to detect risks before development stage.
Technical Leadership
Participate in architecture reviews and technical decision-making.
Promote engineering excellence through code reviews and design reviews.
Collaborate with Architects, Product Owners in data product technical maturation.
Support & Operations (Run Team Collaboration)
Provide support to the Run/Operations team for production pipelines and data products for complex topics.
Troubleshoot and resolve data incidents, failures, and performance issues
Participate in incident management, root cause analysis, and post-mortems.
Ensure proper handover, documentation, and operational readiness of developed solutions.
Contribute to improving runbooks, monitoring, and operational processes.
180 zł - 200 zł an hour
Technical Skills Required
6+ years of experience in data engineering
Strong technical expertise with:
Python (PySpark)
SQL (advanced level)
Hands-on experience with
Azure-based Data Platform
including:
Databricks
Unity Catalog
Apache Airflow
Azure Data Factory
Reporting and BI tools (Power BI, Tableau or similar)
Solid understanding of data
modelling fundamentals , main architectures and design patterns:
Data Warehouse, Data Lake and Data Lakehouse architectures
Medallion architecture
Metadata-driven architecture
ETL/ELT design patterns
Understanding of:
Partitioning, optimization, and performance
Batch and streaming pipelines
Senior Data Engineer - CR 12/11/26 in Warsaw – pracodawca: Margo Group
Margo Group to doskonały pracodawca, który oferuje wyjątkowe możliwości rozwoju zawodowego w dynamicznym środowisku globalnej instytucji finansowej. Nasza kultura pracy opiera się na współpracy i innowacyjności, a pracownicy mają dostęp do szkoleń oraz programów rozwojowych, które wspierają ich karierę. Pracując w naszym zespole, będziesz miał wpływ na kluczowe procesy oraz możliwość pracy z różnorodnymi partnerami, co czyni tę rolę nie tylko satysfakcjonującą, ale i pełną wyzwań.
