Job Description:
- Communication and leadership experience, with experience initiating and driving projects.
- Experience with data sets, Hadoop, and data modernisation tools.
- Experience in SQL or similar languages and in any one cloud platform AWS/Azure/GCP.
- Development experience in at least one object-oriented language (Python, Java, etc.).
- BA/BS in Computer Science, Math, Physics, or other technical fields.
Job Responsibility:
1. Building and maintaining robust modern data architectures
2. Designing efficient data pipelines
3. Ensuring data quality and integrity
4. Scaling data platform solutions and provide thought leadership
5. Expertise in any one of the cloud platforms AWS/Azure/GCP
6. Proficiency in Python/SQL and Data warehousing concepts
Skills:
Strong hands-on experience in Databricks, Apache Spark (PySpark/Scala), and Delta Lake. Expertise in SQL, ETL/ELT pipelines, and data modelling.
Experience with Azure, AWS, or GCP cloud platforms. Knowledge of MLOps, MLflow, and CI/CD best practices.
Experience in workflow orchestration using Databricks Workflows, Airflow, or Prefect.
Understanding of cost optimization, cluster tuning, and performance monitoring in Databricks. Strong leadership, stakeholder management, and mentoring skills.
Experience with data lakehouse architectures and Unity Catalog.
Hands-on with Terraform, Infrastructure-as-Code (IaC), or Kubernetes.
Familiarity with data governance, security, and privacy frameworks.