Key Responsibilities
As an Advanced Data Engineer, you will be part of a team that delivers contemporary analytics solutions for all Honeywell business groups and functions. You will build strong relationships with leadership to effectively deliver contemporary data analytics solutions and contribute directly to business success. You will develop solutions on various Database systems viz. Databricks, Hive, Hadoop, PostgreSQL, etc.
You will identify and implement process improvements – and you don’t like to the same thing twice so you will automate it if you can. You are always keeping an eye on scalability, optimization, and process. You have worked with Big Data before, IoT data, SQL, Azure, AWS, and a bunch of other acronyms.
You will work on a team including scrum masters, product owners, data architects, data engineers, data scientists and DevOps. You and your team collaborate to build products from the idea phase through launch and beyond. The software you write makes it to production in couple of sprints. Your team will be working on creating a new platform using your experience of APIs, microservices, and platform development.
You Must Have
Bachelor's degree in Computer Science, Engineering, Applied Mathematics or related field
8-11 years of data engineering experience
Should have developed and deployed complex big data ingestion jobs in Spark/Informatica BDM/Talend bringing prototypes to production on Hadoop/NoSQL/MPP platforms.
Should have minimum 5 years of hands on experience with Spark, Pig/Hive, etc. and automation of data flow using Informatica, Spark, NiFi or Airflow/Oozie.
Minimum 5 years of experience in developing and building applications to process very large amounts of data (structured and unstructured), including streaming real-time data – Python Spark and Scala. (Kafka, Spark streaming or other such tools).
Hands on experience in Databricks, Cloudera, Hortonworks and/or Cloud (AWS EMR, Azure Data Lake Storage) based Hadoop distributions.
Effective communication skills and succinct articulation
We Value
Minimum 4+ years of experience in working with at least one NoSQL system (HBase, Cassandra, MongoDB etc.).
In-depth knowledge of schema design to effectively tackle the requirement.
Experience in writing complex SQL statements
Experience in working with cloud-based deployments. Understanding of containers & container orchestration (Swarm or Kubernetes).
Experience in building advanced analytics solutions with data from enterprise systems like ERPs, CRMs, Marketing tools etc.
Experience with dimensional modeling, data warehousing and data mining
Good understanding of branching, build, deployment, CI/CD methodologies such as Octopus and Bamboo
Experience working with in Agile Methodologies and Scrum Knowledge of software best practices, like Test-Driven Development (TDD)
Database performance management and API development
Technology upgrade oversight
Experience with visualization software (Tableau, Spotfire, Qlikview, Angular js, D3.js)
Understanding of best-in-class model and data configuration and development processes
Experience working with remote and global teams and cross team collaboration
Consistently makes timely decisions even in the face of complexity, balancing systematic analysis with decisiveness