Senior Data Engineer, Global Converse, ITC

The ideal candidate will have:

1. 5+ years of experience with Databricks, Apache Spark, and cloud-based data integration.
2. Strong Technical expertise with cloud-based platforms, including AWS and or Azure cloud.
3. Strong programming skills in languages like SQL, Python, Java, or Scala.
4. 3+ years' experience with cloud-based data infrastructure and integration leveraging tools like S3, Airflow, EC2, AWS Glue, DynamoDB & Lambdas, Athena, AWS Code deploy, Azure Data Factory, or Google Cloud Dataflow.
5. Experience with Jenkins and other CI/CD tools like GitLab CI/CD, CircleCI, etc.
6. Experience with containerization using Docker and Kubernetes.
7. Experience with infrastructure such as code using tools like Terraform or CloudFormation
8. Experience with Agile development methodologies and version control systems like Git
9. Experience with IT service management tools like ServiceNow, JIRA, etc.
10. Data warehousing solutions, such as Amazon Redshift, Azure Synapse Analytics, or Google BigQuery will be a plus but not mandatory.
11. Data science and machine learning concepts, including TensorFlow, PyTorch, or scikit-learn will be a plus but not mandatory.
12. Strong technical background in computer science, software engineering, or a related field.
13. Excellent collaboration, communication, and interpersonal skills.
14. Experience with data governance, data quality, and data security principles.
15. Ability to lead and mentor junior team members.
16. AWS Certified Solutions Architect or AWS Certified Developer Associate or Azure Certified Solutions Architect certification.

WHAT YOU’LL WORK ON

1. Design and implement scalable data integration solutions using Databricks, Apache Spark, and cloud-based platforms.
2. Develop and implement cloud-based data pipelines using Databricks, Nifi, AWS Glue, Azure Data Factory, or Google Cloud Dataflow.
3. Collaborate with cross-functional teams to deliver high-quality solutions that meet business requirements.
4. Develop and maintain technical standards, best practices, and documentation.
5. Integrate various data sources, including on-premises and cloud-based systems, applications, and databases.
6. Ensure data quality, integrity, and security throughout the integration process.
7. Collaborate with data engineering, data science, and business stakeholders to understand requirements and deliver solutions.