Principal Software Engineer – Data Lakes

Job title:

Principal Software Engineer – Data Lakes

Company

Fivetran

Job description

About the RoleFivetran is building data pipelines to power the modern data stack for thousands of companies.We’re seeking a Principal Software Engineer to lead the technical architecture, design, and development of Fivetran’s Data Lake product offering. Our Data Lake architecture ensures our customers have queryable data without any transformations. We strive to continuously improve our Data Lake solution for scalability, better performance, and ease of customer use.This full-time position is based out of Toronto, Canada.Technologies You’ll Use

• Java, C++, Postgres, Temporal, gRPC
• AWS, GCP, Azure, Kubernetes, Newrelic
• Iceberg, Delta Lake, Parquet, DuckDB

What You’ll Do

• Partner closely with product teams to understand requirements and design cutting-edge new capabilities that go directly into customer’s hands.
• Design, develop, implement, and operate highly reliable large-scale data lake systems in cooperation with a dedicated data lake engineering team.
• Contribute to open-source projects such as DuckDB.
• Embrace Fivetran innovations with open-source standards and toolsets.
• Analyze fault-tolerance and high availability issues, performance and scale challenges, and solve them.
• Ensure operational excellence of the services and meet the commitments to our customers regarding security, reliability, availability, and performance.
• Set technical directions and influence cross-functional teams.

Skills We’re Looking For10+ years of hands-on or research experience with large-scale data-intensive distributed systems, especially in query engines, object storage, data warehouses, data lakes, data analytics, SQL/NoSQL databases, distributed file systems, and data platform infrastructure.Deep understanding of infrastructure & software optimizations and performance engineering to drive significant performance, latency, and availability improvements.Proven track record of leading and delivering large and complicated projects across organizations.Strong development skills in Java and C++.Solid experience with public clouds (AWS, Azure, GCP).Demonstrated knowledge of columnar storage formats.Growth mindset and excitement about breaking the status quo by seeking innovative solutions.Excellent team player who is consistent in making everyone around you better.Strongly prefer an MS or PhD in Computer Science, ideally focusing on database management and/or storage engines.#LI-MD1

Expected salary

Location

Toronto, ON

Job date

Sat, 13 Apr 2024 00:34:58 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (globalvacancies.org) you saw this job posting.