Tasks
· Mandatory: Deep hands-on expertise in Databricks, Python, SQL queries.
· Must be experienced in
o design and implementation of Big Data technologies (Apache Spark, Hadoop ecosystem, Apache Kafka, NoSQL databases)
o working as a Big Data Engineer: query tuning, performance tuning,
o data architecture patterns (Data lakehouse, delta lake, streaming, Lambda/Kappa architecture)
o a full range of data engineering approaches, covering theoretical best practices and the technical applications of these methods.
o building and deploying a range of data engineering pipelines into production, including using automation best practices for CI/CD.
o troubleshooting, and debugging Spark and other big data solutions.
o writing SQL queries.
o databases and analytics technologies Data Warehousing/ETL/ELT, Relational Databases, or MPP
· Must be familiar with
o Data Lifecycle
o Data Mesh
o eXtollo service offerings and onboarding processes
o Azure cloud and creating Azure ADF pipelines.
o data analytics and complex event processing
· Preferred
o Knowledge of Aftersales domain & Data Sources
· Should have good communication (written and verbal)
Qualifications
- Bachelor of Engineering - CSE, IT
- Preferable to have certifications in AZURE, Databricks...