We are seeking an experienced Data Engineer with strong Snowflake skills to join our data engineering team. The ideal candidate will have a solid background in data ingestion, data processing, and data management using technologies such as Snowflake, Snowpark,DBT .
Knowledge on Apache Spark, Hive, and HDFS would be good to have.. You will be responsible for migrating and testing data pipelines and ensuring efficient data storage and processing. In addition, you will work with CI/CD tools, such as Jenkins, and schedulers like CTRL-M, to automate data workflows.
Profile required
· At least 3 to 7 Years of experience on big data platform and at least 2 years of experience in implementing DWH on Snowflake
· Proven experience with cloud platforms preferable Azure particularly on data services
· Good understanding of distributed computing frameworks like Apache Spark, Hadoop etc
· Perform Analysis on existing data storage systems Big Data & development of data solutions in Snowflake
· High Proficiency in SQL and at least one of the following languages Scala / Python &
· Experience in working on migrating data from on-premise databases preferably Big Data platformto Snowflake
· Expertise in building robust ELT/ETL processes , performance tuning of the data pipelines in Snowflake and should be able to trouble shoot the issues quickly
· Strong Knowledge on Integration concepts and design best practices
· Data Modeling & data integration, Advanced SQL skills for analysis, standardizing queries
· Proven experience in managing and mentoring data engineering teams
· Excellent interpersonal skills, with the ability to work across teams and communicate effectively with technical and non-technical stake holders
· Strong analytical and trouble shooting skills with proven ability to find solutions in complex data environments
Technical Skills Required
· DB : Snowflake, Snowpark
· CSP : Azure
· Big data with devops environment : HDFS, YARN, AWX, API file transfer, Hive Querying, Jenkins, CTRL-M
· Language: Scala or Python
· Source code : Git on github.
· OS : Linux and good proficiency on shell scripting.
· Methodology : Agile via Scrum/Kanban (tool Jira)