Principal Platform Engineer – Azure Data & Analytics

Key Responsibilities:

Technology Leadership:

• Work independently as a platform engineer to and lead design , development and evolution of Azure Cloud Platform operations related Data Analytics and AI Services.
• Define and manage cloud infrastructure services through automation, CI/CD and best practices.
• Troubleshoot data engineering related services like databricks pipelines, ADF, ADLS Datalakes and Azure cloud related issues

Architecture Review:

• Communicate with stakeholders to understand requirements and propose cost effective and performant architecture design.
• Present platform architecture, roadmap, plans, status, and risk to various stakeholders, including senior engineers.
• Champion Devops best practices and ensure the data engineering teams follow all the recommended architecture guidelines while delivering data products

Platform Engineering:

• Champion scalable and cost-effective data and advanced analytics platforms, evangelize best practices for building cutting-edge data and analytics ecosystem.
• Work closely with senior engineers and various stakeholders to understand requirements, create roadmaps, keep track of key milestones, and take accountability for the delivery of new platforms developments.

Continuous improvement:

• Develop and enable processes that allow for continuous improvement through structured feedback, error/variation detection, performance measurements, data security evaluations, and data quality measures.

Project Execution & Management:

• Demonstrate a combination of business focus, strong analytical and problem-solving skills, and programming knowledge to be able to quickly iterate through PoC in the discovery phase of the project to a enterprise scale production implementation of a GxP solution

Team Management:

• Work with vendors and technology partners to select and implement new data solutions and provide operational support for data solutions already deployed.
• Work with a team of data architects and data engineers that design, develop, and maintain data pipelines, data warehouses, and other data infrastructure.

Collaboration & communication:

• Collaborate and work closely with executives, stakeholders and business teams to effectively communicate architecture strategy & clearly articulate the business value.

Qualifications for Principal Engineer – Data and Analytics

• BS or MS in engineering, sciences, or equivalent relevant experience required.
• Overall 12 to 14yrs of experience in IT
• 6+ years of experience in building and managing data and analytics applications is required
• Demonstrated expertise on Azure, especially on data services like Databricks, ADF etc.
• Proficiency in Python programming and Data engineering stacks
• Suitable candidate be able to demonstrate strong experience in the following areas.

Technology Experience

Candidate must have strong experience in the following technology areas –

• Azure Administration and Data Engineering –
• Proficient and certified in Azure administration.
• Proficient on modern Data engineering stack such as Data bricks, DBT, Delta Lake, Spark
• Hands-on experience working with Azure Functions, logic apps,
• Power BI, Power Apps, Azure Security controls.
• Working knowledge of Devops pipelines using tools like GIT and Azure DevOps Pipelines
• Good understanding of working with structured as well as unstructured data
• Project Management & Communication Skills
• Strong project management understanding with experience in executing projects using Agile delivery framework
• Sound knowledge of project management tools such as ADO, Jira, MS Projects
• Strong analytical, organizational, interpersonal, and time management skills
• Excellent communication (written and verbal) and interpersonal skills
• Programming Skills
• Proficient in SQL and Python or Java programming languages
• Proficient in implementing tools and systems using Kubernetes .
• Proficiency in deploying and debugging applications on Linux OS

Preferred Qualifications

In addition to above listed skills, ideal candidate may have one or more of below qualifications -

• Solution Architecture
• Designing solution architecture and present & defend the architecture in multiple architecture review forums
• Technology Consulting experience in Cloud and Data Domain
• Strong problem solving and analytical skillset
• Certifications:
• Azure Data Engineer, Developer, Devops, Azure Solution Architect or equivalent certification preferred
• Databricks/Data Engineering Certification (Good to have)
• Compliance Knowledge:
• Experience in working with Cloud Security Controls and DevSecOps best practices.
• Good To Have:
• Hands on experience on Spark, Synapse, Azure Data Factory, CI/CD, Automated Testing, Data Lakes, DWH, Big Data, Unity Catalog