Roles and Responsibilities
We are building a Data Engineering Platform using Azure and Data bricks, which requires good coding skills on Python + PySpark along with Data bricks and Azure services understanding. Because this is a platform so deployment and testing through code is a must – Pytest and CI/CD using Jenkins are needed.
Skills and Qualifications
- 6+ years of experience in Data engineering.
- Strong Python Developer with good software design fundamentals.
- Experience of CI/CD is must (preferably Jenkins).
- Strong Hands-on Pytest experience.
- Strong Data bricks PySpark, Delta Tables, Delta Live Tables.
- Experience in best Practices of Data Engineering, Unity Catalog (Experience or at least theoretical understanding).
- Azure Cloud Understanding: Working experience on ADLS, Key Vault, Azure Config Services, IAM.
- BI platform collaboration experience, preferably Power BI.
- Terraform understanding is a plus.