Senior Data Engineer

Overview

As a Senior Data Engineer at nZero, you will play a pivotal role in designing, developing, and maintaining scalable data pipelines and storage tools to support our platform. You will work closely with our cross-functional teams, including Data Scientists, Analysts, and Software Engineers, to ensure seamless data flow and transformation from various sources to enable data-driven decision-making. Your expertise in AWS, dbt, and Python will be critical in building efficient and sustainable data processing solutions tailored to our unique needs.

Responsibilities:

• Design and implement scalable data pipelines, including data ingestion, transformation, and integration, using AWS services such as Amazon S3, Glue, RDS, Aurora and Redshift.
• Collaborate with stakeholders to understand data requirements and develop data models aligned with nZero ‘s business goals.
• Build and maintain ETL/ELT workflows using dbt (Data Build Tool) and Prefect to transform raw data into structured datasets for analysis, reporting, and sustainability tracking purposes.
• Optimize and tune data pipelines for performance, scalability, and reliability
• Implement data quality controls and validation processes to ensure the accuracy and integrity of nZero’s data
• Monitor data pipelines, perform debugging and troubleshooting, and implement proactive measures to ensure data availability, accuracy, and efficient resource utilization.
• Stay up to date on the latest advancements in data engineering technologies, tools, and best practices, and propose their adoption when they align with nZero’s objectives.

Requirements:

• Bachelor’s degree in Computer Science, Engineering or Information Systems or strong work experience in a related technology
• Proven experience as a Data Engineer or similar role, with a strong understanding of data engineering concepts, data modeling, and database systems.
• Familiarity or Hands-on experience with Data Lakehouse architectures and implementing them.
• Hands-on experience with AWS data services, including S3, Glue, RDS, Aurora, Redshift, and related tools and technologies.
• Hands-on experience developing within a CI/CD data pipeline and deploying code into different environments
• Proficiency with Git and source control
• Proficiency in DBT (Data Build Tool) for data transformation and modeling.
• Strong programming skills in Python and familiarity with related libraries and frameworks.
• Experience with SQL and working knowledge of relational databases and/or columnar databases.
• Familiarity with data orchestration techniques and tools, such as Prefect.
• Familiarity with Datadog or similar logging and alerting platforms.
• Knowledge of data warehousing concepts and dimensional modeling.
• Strong problem-solving and analytical skills, with the ability to work in a fast-paced and collaborative environment.
• Excellent communication and interpersonal skills, with the ability to translate complex technical concepts to non-technical stakeholders
• Passion to make an impact and do some awesome work.

About You:

1. Action Oriented: You are driven by a strong need to act when confronted with a problem, knowing that in most situations the “perfect” answer does not exist yet you are able to find the best solution at that time.
2. Comfortable with ambiguity: You are comfortable handling and managing moderate levels of ambiguity in your role. We are a young start-up in a new market, creating products and services which have never existed before, which often requires blazing new trails with uncertain outcomes.
3. Passionate: You care about the quality of the things you produce and the problems you solve. At nZero, we are a passionate team who care deeply about the climate challenges we are tackling.
4. Inclusive: You understand that accepting and integrating a diverse set of views creates growth and opportunity.