Senior Azure Solution Architect/Engineering Lead

Core Skills: Azure Databricks, PySpark, Spark, SQL, Azure Synapse, Kafka, Big Data, Python, Java, Azure Data Factory (ADF)

Professional Summary

Senior Data Architect with 15+ years of experience designing, building, and scaling enterprise-grade data platforms. Proven expertise in cloud-native big data architectures on Azure, real-time and batch data processing, and high-performance analytics systems. Strong background in data modeling, distributed computing, and end-to-end data pipeline orchestration, with hands-on leadership in large-scale enterprise and mission-critical projects.

Technical Expertise

Cloud & Data Platforms

Azure Databricks (Lakehouse architecture, Delta Lake, Unity Catalog)
Azure Synapse Analytics (Dedicated & Serverless Pools)
Azure Data Factory (ADF) for ELT/ETL orchestration
Azure Data Lake Storage Gen2

Big Data & Distributed Processing

Apache Spark (Core, SQL, Structured Streaming)
PySpark performance tuning & optimization
Apache Kafka (real-time ingestion, streaming pipelines)
Large-scale batch and streaming data architectures

Programming & Query Languages

Python (advanced data engineering, automation, frameworks)
Java (Spark internals, Kafka consumers/producers, microservices)
SQL (complex analytics, query optimization, warehouse design)

Data Architecture & Design

Lakehouse, Lambda, and Kappa architectures
Dimensional modeling & data warehouse design
Data governance, security, and lineage
Scalability, fault tolerance, and cost optimization
Technical depth in designing and building data platforms for high volume data processing with low latency
Strong Expertise in Azure, Databricks, Python, Pyspark, SparkSQL, CI-CD (Jenkins, Github), Obs (Dynatrace),
Good to have : Java, Kafka , Casandra, Microservices, API, Flink, ZeroMQ, Protocol Buffers
Experience in recruiting , people leadership for managing team of 40+ , govern delivery and also act as technical mentor to set processes to bring innovation, goals to derive business outcomes
Act as Module lead for India operations, liaise with NA teams to drive the objectives

Key Responsibilities & Achievements

Architected and delivered enterprise-scale Azure Lakehouse platforms supporting petabyte-scale data.
Designed real-time streaming solutions using Kafka and Spark Structured Streaming.
Led migration of on-premise data warehouses to Azure Synapse + Databricks, improving performance and reducing cost.
Built reusable PySpark frameworks for data ingestion, transformation, and validation.
Optimized Spark jobs through partitioning, caching, and memory tuning to achieve significant runtime improvements.
Implemented end-to-end ADF pipelines with CI/CD and parameter-driven orchestration.
Mentored senior and junior engineers; provided architectural guidance across multiple teams.
Collaborated with business stakeholders to translate analytics requirements into scalable technical designs.

Leadership & Soft Skills

Technical leadership and architectural decision-making
Cross-team collaboration and stakeholder communication
Design reviews, code quality standards, and best practices
Agile/Scrum execution in large enterprise environments
Functional experience of capital markets, securities processing will be advantage

Engineering Lead/Azure Solution Architect