Job Description:
Team Leadership:
Leading and managing a team of data engineers, providing technical guidance, mentorship, and fostering a collaborative environment.
Data Pipeline Development:
Designing, developing, and implementing scalable data pipelines for ingesting, processing, and storing data from various sources.
Data Architecture:
Designing and implementing robust and scalable data architectures, including data models, data lakes, and data warehouses.
Data Quality and Governance:
Ensuring data quality through data validation, cleansing, and implementing data governance policies.
Performance Optimization:
Optimizing data systems for performance, scalability, and cost-effectiveness.
Collaboration:
Working closely with data scientists, analysts, and other stakeholders to understand their needs and ensure data is available for their use.
Technical Guidance:
Providing technical guidance and mentorship to junior data engineers, and staying up-to-date with the latest data engineering technologies and trends.
Project Management:
Managing data engineering projects from inception to completion, including setting goals, defining scope, and ensuring timely delivery.
Infrastructure Management:
Building and maintaining the infrastructure required for optimal data processing and storage.
Skills Required:
Pyspark,Python,AWS,Azure,DataBricks,DevOps,Airflow
Airflow, PowerBI,Autosys,GENAI