Azure Databricks Consultant Location: 100% remote We are seeking a skilled Azure Databricks Engineer to join our team. The ideal candidate will have hands-on experience with Azure Databricks and a strong background in big data technologies. This role involves designing, implementing, and optimizing data processing workflows and analytics solutions within the Azure Databricks environment. Key Responsibilities:
- Data Engineering: Design and develop scalable data pipelines using Azure Databricks to handle large volumes of data from various sources.
- Data Integration: Integrate data from multiple sources including cloud storage (Azure Blob, Azure Data Lake), databases, and APIs.
- Performance Optimization: Optimize data processing and querying performance for both batch and streaming data.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand requirements and deliver effective solutions.
- Monitoring and Troubleshooting: Implement monitoring solutions for data pipelines and troubleshoot issues related to data processing and performance.
- Documentation: Create and maintain documentation for data pipelines, workflows, and best practices.
- Innovation: Stay up-to-date with the latest advancements in Azure Databricks and related technologies, and recommend improvements to existing processes and tools.
Required Qualifications:
- Experience: 3+ years of experience in data engineering or related roles with a focus on Azure Databricks.
Technical Skills:
- Proficiency in Azure Databricks, Spark, and related technologies.
- Experience with programming languages such as Python, Scala, or SQL.
- Knowledge of data storage solutions (e.g., Azure Blob Storage, Azure Data Lake).
- Familiarity with data modeling and ETL processes.
- Education: Bachelor's degree in Computer Science, Engineering, Data Science, or a related field, or equivalent work experience.
Soft Skills:
- Strong analytical and problem-solving skills.
- Excellent communication and teamwork abilities.
- Ability to manage multiple tasks and priorities effectively.
Preferred Qualifications:
- Certifications: Azure Data Engineer Associate or similar certifications.
- Experience with Data Warehousing: Knowledge of data warehousing concepts and tools (e.g., Azure Synapse Analytics).
- Cloud Experience: Experience with other Azure services (e.g., Azure SQL Database, Azure Data Factory) is a plus.
#DICE ...@c4techservices.com