Scala Spark Lead 15+ years experience Required
: Job Details :

Company DescriptionJob Description Job Title: Scala Spark Lead 15+ years experience Required Location: Broadway, New York, NY 10036 (3 days onsite per week) Duration: 10/08/2024 - 12/31/2024 (with potential extension) Job Role: We are seeking a highly skilled Scala Spark Lead with over 10 years of experience to join our dynamic team. In this role, you will lead the design, development, deployment, and optimization of data processing applications using Spark and Scala. Key Responsibilities: Design, develop, and maintain data processing applications using Scala and Apache Spark. Collaborate with cross-functional teams to gather data requirements and design efficient solutions. Troubleshoot and optimize Spark applications for performance issues, ensuring optimal data processing. Implement test-driven deployment practices to improve the reliability and quality of applications. Deploy artifacts to higher environments, ensuring smooth transitions and production releases. Work in an Agile environment, contributing to sprint planning, development, and timely delivery of high-quality solutions. Provide essential support for production batches, troubleshooting issues to meet business-critical needs. Mandatory Skills & Experience: Big Data: 7+ years Scala: 8+ years Apache Spark: 7+ years Core Java: 5+ years 10+ years of experience in software engineering, focusing on Big Data, Scala, and Apache Spark. Strong understanding of design patterns and data structures for efficient data processing. Familiarity with database concepts and SQL. Proficiency in troubleshooting and debugging Spark applications for optimal performance. Good to Have Skills: Knowledge of Java and Snowflake. Familiarity with Python. Understanding of Databricks platform. Experience with test-driven deployment practices. Knowledge of DevOps practices.