What you will do
- Build a data platform using big data processing and warehousing technologies.
- Build, maintain, and ideate on frameworks and tools required to ingest, store, process and serve data for global scale, data-intensive systems.
- Build data pipelines to publish reports and to power dashboards for ALM clients.
- Collaborate with cross-functional teams to determine key interfaces for your projects and build them with quality and agility.
- Do analysis and present key findings, insights and concepts to key influencers and leaders to help them make data driven decisions on business and product roadmap.
- Support Machine Learning engineers and Data Scientists with their dataset needs for training the ML or statistical models and evaluating them.
- Be aware of the latest open-source projects and technological breakthroughs in the data engineering space and explore them to build better and more effective solutions.
- Drive innovation through research and experimentation, encouraging an environment where new ideas can thrive.
- Mentor junior engineers on the team
What you need to succeed
- A bachelor's degree in computer science or related fields.
- 8-12 years of experience as a server-side developer, with at least 4 years in Data Engineering, with a consistent track record of designing, implementing, and delivering large scale, high-quality solutions.
- Solid understanding and programming skill in languages such as Java or Python and frameworks and libraries based on them with a focus on data engineering.
- Strong expertise and familiarity with various Big Data technologies and frameworks including Delta Lake, Apache Hive, Datahub, Apache Spark, Apache Flink, Apache Storm, Trino, and AWS EMR.
- Strong familiarity with the concepts of RDBMS, Data Lake, Data Warehouse, Data Lakehouse, and Medallion Architecture.
- Knowledge of at least one workflow orchestration tools such as Airflow or Oozie.
- Experience working with ML engineers, Data Scientists, product and engineering leadership on their data and analysis needs.
- Experience working with the DevOps teams to drive operationally excellent infrastructure.
- Excellent problem-solving skills, with a consistent track record of delivering innovative solutions.
- Ability to thrive in a collaborative, inclusive, and diverse workplace, embracing different perspectives and ideas.
- Being comfortable with the ambiguity and having the ability to adapt to evolving priorities.
- Be a role model and a mentor for the team by doing diligent code and design reviews, by doing knowledge sharing sessions and showcasing sound technical decision making in day-to-day work.
- Being ahead of the curve by familiarizing yourself with the latest and the best in the technological landscape and turning them into effective solutions for our customers.