B
Senior Data Engineer
B
- Posted a month ago
- Be among the first 10 applicants
Job Description
Job Duties
- Designs, builds, and operates industrialized data ingestion and transformation pipelines from source systems through the silver layer of a medallion architecture, using Databricks (Spark, Unity Catalog, workflows) as the core platform.
- Serves as the team's Databricks subject matter expert — data modeling in Delta, job and cluster design, performance and cost tuning, and Unity Catalog governance patterns.
- Guides, coaches, and technically mentors other Data Engineers; conducts design and code reviews to ensure architecture, platform, and solution standards are followed.
- Builds and maintains pipelines across multiple clouds — primarily AWS (Glue, Step Functions, Redshift, Aurora, Transfer Family, S3) and Azure (Data Factory, ADLS) — and plans the migration of native cloud pipelines to Databricks as a destination technology.
- Defines and enforces the guardrails, standards, and reusable patterns that business units follow when building their own projects on the platform, balancing self-service velocity against governance requirements.
- Partners with the BI and analytics teams to deliver curated, well-modeled, well-documented data products fit for reporting and for conversational query experiences such as Databricks Genie Space.
More Info
Key Skills
Unity Catalog
Transfer Family
Aurora
ADLS
About Company
Skills:
Databricks, Excel, Azure, Sql, Python
Skills:
data engineering , Scala, Spark, Kafka, AWS, Real-time Processing
Skills:
snowflake , Cloud-based Solutions, Python, Sql, S3, Pyspark, Big Data Technologies, AWS, Etl, Sns, Data Modeling, Glue, Performance Optimization, AI-assisted Development Workflows
Skills:
snowflake , Snowpipe, AWS Glue, Openflow, Sql, Python, Azure DevOps, ELT, Etl, dbt, fivetran
Skills:
Azure Data Factory, SQL Server, Data Warehouse, Sql, ELT, Etl, Pipeline monitoring and troubleshooting, Data pipelines, Data load error analysis, Azure SQL Database