We are looking for creative and enthusiastic Data Engineers to join our team in building the best Data Platform on the street. We're responsible for managing the flow of data into the firm, maintaining the data lake, creating analytics-ready datasets, and building the APIs that make everything accessible to our clients. Our singular goal is to help our investment teams use data to make better investment decisions. Our analysts and systematic trading teams rely on us to provide analytics-ready datasets. For each dataset we must consider the implications of point in time storage, optimize for our users' access patterns, and create useful aggregations/slices. Our ideal candidate will have experience with storing, transforming, and modeling big data. In this role, you will:
- Develop cloud-first data ingestion processes using Python, SQL, and Spark
- Engineer data models and infrastructure for a wide variety of market and alternative datasets
- Design and build services and plugins to enhance our Data Acquisition Platform
- Maintain alerting systems to ensure smooth day-to-day operations for hundreds of datasets
- Author tests to validate data quality and the stability of the platform
- Investigate and defuse time-sensitive data incidents
- Communicate with data providers to onboard new datasets and troubleshoot technical issues
- Evangelize best practices to our partners throughout the firm
- Work directly with Analysts, Quants, and Portfolio Managers to understand requirements and provide end-to-end data solutions
WHAT YOU'LL BRING
- Bachelors/master's degree in computer science or a related field
- Strong analytical, data, and programming skills (Python/SQL/NoSQL)
- 3+ years of experience with at least one of Spark/Hive/Hadoop
- 2+ years of experience orchestrating pipelines with a technology like Airflow/Luigi/Oozie/Nifi
- 1+ years of experience with cloud technologies (AWS / Azure / Google Cloud)
- Solid understanding of time series data and temporal queries
- Experience with large data sets and techniques to architect them for performance
- Ability to understand and contribute to our existing data system software
- Aptitude for designing infrastructure, data products, and tools for Data Scientists a plus
- Financial industry experience is a plus
- Strong oral and written communication skills, most importantly, must be a team player