Title: Microsoft Azure/Fabric Data Engineer Location: Remote Duration: 3 monthsDescription: Data Tech team at Client is looking for a Data Engineer to join a diverse team dedicated to providing best in class data services to our customers, stakeholders and partners. As a part of our organization, you will work with our ML Engineers, Data Scientist and various BUs to define solutions for operationalizing data-driven decision making in a cost effective and scalable manner.Qualifications:
- Bachelor's degree in computer science, Software Engineering, MIS, or an equivalent combination of education and experience.
- Experience implementing and supporting data lakes, data warehouses, and data applications on AWS or Microsoft Azure for large enterprises (Fabric is plus)
- Programming experience with Python, Spark and SQL.
- Experience with Microsoft Azure services, specifically Data Factory, ADLS Gen2, Synapse Analytics, and Synapse Database, KQL.
- Experience in system analysis, design, development, and implementation of data ingestion pipelines in AWS or Azure.
- Solid experience implementing solutions on AWS or Azure-based data lakes (preferred).
- Knowledge of ETL/ELT processes.
- Experience with end-to-end data solutions (ingest, storage, integration, processing, access) on AWS or Azure (preferred).
- Architect and implement CI/CD strategies for EDP.
- Implement high-velocity streaming solutions.
- Migrate data from traditional relational database systems, onprem to Azure technologies (preferably in Onelake using Fabric technologies - (preferred).
- Develop data-based APIs
- Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-cases.
- Any Cloud Certification preferred (Azure / AWS preferred)
- Knowledge of Microsoft Fabric technology focusing on cost and security considerations.
Requirements:
- 5+ years of experience as a Data Engineer/Datalakes .
- Experience developing business applications using NoSQL/SQL databases.
- Experience working with Object stores and Structured/semi-structured/unstructured data is a must.
- Experience with Microsoft Azure services - Data Factory, ADLS Gen2, Synapse Analytics, and Synapse Database.
- Develop CI/CD pipeline using Terraform and github
- Microsoft Fabric is plus.
Responsibilities:
- Designing, building, and maintaining efficient, reusable, and reliable architecture and code.
- Ensure the best possible performance and quality of high-scale web applications and services.
- Participate in architecture and system design discussions.
- Independently perform hands-on development and unit testing of the applications.
- Collaborate with the development team and build individual components into complex enterprise web systems.
- Work in a team environment with product, frontend design, production operation, QE/QA, and cross-functional teams to deliver a project throughout the whole software development cycle.
- Responsible for identifying and resolving any performance issues.
- Keep up to date with new technology developments and implementations.
- Participate in code reviews to ensure standards and best practices are met.1