Senior Data Engineer

arrow

New York / $150000 - $180000 annum

INFO

Salary
SALARY:

$150000 - $180000

Location

LOCATION

New York

Job Type
JOB TYPE

Permanent

Senior Data Engineer
Technology
United States (Remote)
$150,000 - $180,000 + equity

The Company
Our client is revolutionizing operational safety in critical industries. They specialize in leveraging computer vision to provide real-time insights for the Construction industry. Their mission is to eliminate failures due to inadequate visual inspection, ensuring safety, reliability, and compliance.

Role Description
As a Senior Data Engineer, you will play a pivotal role in designing and optimizing scalable data pipelines that power machine learning and analytics applications. You will be responsible for architecting a robust data infrastructure, collaborating with ML engineers, and ensuring efficient storage and processing of large-scale industrial and vision data. This is an opportunity to make a significant impact on a rapidly growing team that values technical excellence and innovation.

Key Responsibilities

  • Architect and build scalable data pipelines using Dagster to transform and move data for analytics and ML.
  • Design and optimize storage solutions for large-scale industrial and vision data, ensuring efficient retrieval and processing.
  • Develop data ingestion frameworks for real-time and batch processing of images, video, and metadata.
  • Work with ML engineers to structure data effectively for model training and experimentation.
  • Deploy and manage data pipelines in Kubernetes-based environments.
  • Automate CI/CD workflows for data infrastructure using GitLab CI/CD.
  • Manage AWS-based data infrastructure, leveraging Terraform for Infrastructure as Code.
  • Implement data governance best practices, ensuring data quality, lineage tracking, and metadata management.
  • Optimize batch and real-time processing frameworks for scalability and performance.
  • Provide technical leadership, establishing best practices and guiding future data scaling strategies.

Requirements

  • 5+ years of experience in data engineering with a focus on scalable, production-grade data infrastructure.
  • Strong expertise in Python (Pandas, PyArrow, Dask, etc.).
  • Hands-on experience with data orchestration tools (Dagster, Prefect, or Airflow).
  • Proficiency in Kubernetes for data pipeline orchestration.
  • Experience deploying infrastructure via Terraform (or similar IaC tools).
  • Cloud expertise, preferably AWS, including S3, EKS, Lambda, Glue, and RDS.
  • Experience with streaming and event-driven architectures such as Apache Ray Core, Kafka, Kinesis, Pulsar, or Storm.
  • Strong database skills, including SQL, NoSQL, and columnar storage (Postgres, BigQuery, ClickHouse).
  • CI/CD expertise with GitLab for managing automated data pipeline deployments.

The Benefits
As a Senior Data Engineer, you will receive a base salary of $150,000 to $180,000 plus benefits like equity, health, dental and vision.

How to Apply
To apply, please submit your resume via the link on this page.

Key Words
Python, Dagster, Kubernetes, Terraform, AWS, Data Pipelines, CI/CD, Streaming Data, Apache Ray, Kafka, Computer Vision, Machine Learning, SQL, Big Data, Data Engineering

CONTACT

SIMILAR
JOB RESULTS

4k-Harnham_DA copy
CAN’T FIND THE RIGHT OPPORTUNITY?

STILL
LOOKING?

If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.