Senior Data Engineer | Embodied AI

We are partnering with a US-based embodied AI company to hire a Senior Data Engineer.

This is a hands-on, founding-team role suited to someone who has personally built and operated petabyte-scale data platforms in production.

Job Responsibilities

• Build and scale a PB-scale data lake / lakehouse for multimodal spatial data — object storage layout, partitioning, open table formats, catalogs, and metadata

• Own data versioning and lineage: branches, commits, reproducible dataset snapshots, and provenance from raw capture to customer delivery

• Build high-throughput, fault-tolerant ingest pipelines — upload, validation, checksumming, content-addressed deduplication, and promotion through bronze → silver → gold

• Build and operate large-scale distributed processing across CPU and GPU fleets — video decode, frame extraction, sync and calibration, large-scale 3D and spatial data processing, and model-based auto-labeling

• Build and maintain data pipelines at scale: idempotent jobs, retries, backfills, partitioned reprocessing, and clean failure recovery

• Build the dataset curation and query layer — metadata indexing, SQL over the lake, and search so researchers and customers can slice data without copying it

• Keep GPUs fed: large-scale ML training data formats, caching, prefetching, and efficient cross-region data movement

• Own platform reliability, observability, and cost on AWS — S3 tiering, lifecycle, egress, spot and right-sizing, capacity planning, and alerting

• Contribute to infrastructure-as-code and CI/CD; write design docs and runbooks so others can operate the platform without tribal knowledge

• Work closely with data capture, labeling, and QA teams to translate workload requirements into platform capabilities

Job Requirements

• B.S. or M.S. in Computer Science, Software Engineering, or a related field, or equivalent practical experience

• 6+ years in data engineering, data platform, or distributed systems engineering

• Hands-on experience building and operating a data platform at petabyte scale in production — able to speak concretely to volumes, throughput, object counts, and failure modes personally handled

• Deep object storage experience (S3 or equivalent): key/prefix design, throughput limits, small-file problem, lifecycle tiering, and cost control

• Production experience with open table formats — Iceberg, Delta Lake, or Hudi; catalogs such as Glue, Nessie, or similar

• Strong experience with distributed processing frameworks — Spark, Ray, Flink, or Dask — tuned for large production jobs

• Experience with workflow orchestration (Airflow, Argo, Prefect, or similar) with strong discipline around idempotency, retries, and data contracts

• Strong Python plus at least one of Go, Rust, C++, Java, or Scala

• Solid AWS infrastructure skills — EC2, EKS/ECS, S3, IAM, VPC — and Terraform or equivalent IaC

• Production on-call experience for a data platform; care deeply about uptime, correctness, and cost

www.dadaconsultants.com

Licence Number: 18S9037

EA Registration Number: R23112003

Business Registration Number: 201735941W