Key Responsibilities:
- Data Pipeline Development: Design, develop, and maintain robust data pipelines using GCP services like Dataflow, Dataproc, ensuring high performance and scalability.
- Google Big Query Expertise: Utilize your hands-on experience with Google Big Query to manage and optimize data storage, retrieval, and processing.
- SQL Proficiency: Write and optimize complex SQL queries to transform and analyze large datasets, ensuring data accuracy and integrity.
- Python Programming: Develop and maintain Python scripts for data processing, automation, and integration with other systems and tools.
- Data Integration: Collaborate with data analysts, and other stakeholders to integrate data from various sources, ensuring seamless data flow and consistency.
- Data Quality and Governance: Implement data quality checks, validation processes, and governance frameworks to maintain high data standards.
- Performance Tuning: Monitor and optimize the performance of data pipelines, queries, and storage solutions to ensure efficient data processing.
- Documentation: Create comprehensive documentation for data pipelines, processes, and best practices to facilitate knowledge sharing and team collaboration.
Minimum Qualifications:
- Proven experience (minimum 6 – 8 yrs) in Data Engineer, with significant hands-on experience in Google Cloud Platform (GCP) and Google Big Query.
- Proficiency in SQL for data transformation, analysis and performance optimization.
- Strong programming skills in Python, with experience in developing data processing scripts and automation.
- Proven analytical skills with the ability to interpret complex data and provide actionable insights.
- Excellent problem-solving abilities and attention to detail.
- Strong communication and collaboration skills, with the ability to work effectively in a team enviro
Desired Skills :
- Experience with Google Analytics data and understanding of digital marketing data.
- Familiarity with other GCP services such as Cloud Storage, Dataflow, Pub/Sub, and Dataproc.
- Knowledge of data visualization tools such as Looker, Tableau, or Data Studio.
- Experience with machine learning frameworks and libraries.