JobRialto
Location: Louisville,KY, USA
Date: 2024-11-25T06:26:45Z
Job Description:
Job Summary We are seeking an experienced Azure Databricks Developer with 7+ years of professional experience in big data engineering and the Azure data ecosystem. This role will lead the transition from Azure Synapse Analytics to Databricks, leveraging advanced Python/SQL skills, deep expertise in Azure technologies, and a proven track record of delivering scalable and secure data solutions. Key Responsibilities •Migrate and optimize workloads from Azure Synapse Analytics to Databricks. •Develop and implement big data engineering systems using Python, SQL, and Databricks. •Create medallion architecture (bronze, silver, gold) for structured and unstructured data. •Automate workloads using tools like Databricks workflows, Prefect, or CI/CD pipelines. •Ensure data security through schema, table, and row-level security, as well as dynamic data masking. •Develop monitoring, logging, and alerting systems for production-grade data pipelines. •Work with credential management systems like Azure Key Vault or HashiCorp. •Collaborate with cross-functional teams to translate business requirements into technical solutions. •Analyze data and create impactful visualizations using Power BI. •Integrate solutions with additional cloud technologies such as ServiceNow, Splunk, Dynatrace, or Azure Log Analytics. Required Qualifications •7+ years of experience in big data engineering. •5+ years' programming experience in Python and SQL at an expert level. •5+ years' experience in the Azure data ecosystem, including Databricks and Azure Synapse Analytics. •3+ years' experience designing big data systems with medallion architecture. •2+ years' experience in automating workloads and ensuring data security. •2+ years' experience with source/version control systems like GitHub or Azure DevOps. •Proven ability to independently manage big data engineering projects. Preferred Qualifications •Proficiency in Power BI, including Star Schema, DAX, and performance optimization. •Experience with multiple cloud technologies like ServiceNow, Splunk, or Dynatrace. •Strong analytical and problem-solving skills for handling large and complex datasets. •Expertise in credential management and automated response systems. Certifications •Microsoft Certification (DP-203) •Databricks Certified Data Engineer Professional Education: Bachelors DegreeCertification: Microsoft Certified: Azure Data Engineer Associate , Databricks Certified Data Engineer Professional
Apply Now!