Big Data Engineer

What you will do

1. Build a data platform using big data processing and warehousing technologies.
2. Build, maintain, and ideate on frameworks and tools required to ingest, store, process and serve data for global scale, data-intensive systems.
3. Build data pipelines to publish reports and to power dashboards for ALM clients.
4. Collaborate with cross-functional teams to determine key interfaces for your projects and build them with quality and agility.
5. Do analysis and present key findings, insights and concepts to key influencers and leaders to help them make data driven decisions on business and product roadmap.
6. Support Machine Learning engineers and Data Scientists with their dataset needs for training the ML or statistical models and evaluating them.
7. Be aware of the latest open-source projects and technological breakthroughs in the data engineering space and explore them to build better and more effective solutions.
8. Drive innovation through research and experimentation, encouraging an environment where new ideas can thrive.

What you need to succeed

1. A bachelor's degree in computer science or related fields.
2. At least 8 years of experience as a server-side developer, with at least 4 years in Data Engineering, with a consistent track record of designing, implementing, and delivering large scale, high-quality solutions.
3. Solid understanding and programming skill in languages such as Java or Python and frameworks and libraries based on them with a focus on data engineering.
4. Strong expertise and familiarity with various Big Data technologies and frameworks including Delta Lake, Apache Hive, Datahub, Apache Spark, Apache Flink, Apache Storm, Trino, and AWS EMR.
5. Strong familiarity with the concepts of RDBMS, Data Lake, Data Warehouse, Data Lakehouse, and Medallion Architecture.
6. Knowledge of at least one workflow orchestration tools such as Airflow or Oozie.
7. Experience working with ML engineers, Data Scientists, product and engineering leadership on their data and analysis needs.
8. Experience working with the DevOps teams to drive operationally excellent infrastructure.
9. Excellent problem-solving skills, with a consistent track record of delivering innovative solutions.
10. Ability to thrive in a collaborative, inclusive, and diverse workplace, embracing different perspectives and ideas.
11. Being comfortable with the ambiguity and having the ability to adapt to evolving priorities.
12. Be a team player by doing diligent code and design reviews, by doing knowledge sharing sessions and showcasing technical expertise in day-to-day work.
13. Keeping pace with the latest technologies in the big data engineering space and bringing new ideas to solve problems more efficiently.