Primary Scope: Databricks architecture, migration assessment and design, lakehouse, integration, security, performance, and stakeholder solutioning.
Own the end-to-end architecture and solution design for enterprise data-platform migration programmes, with a strong focus on Cloudera/Hadoop-to-Databricks modernisation and banking-grade security, governance, resilience and performance.
Key Responsibilities
- Lead discovery and current-state assessment of data platforms, workloads, integrations, security controls and operational dependencies.
- Define target-state Databricks Lakehouse architecture, migration waves, workload disposition and coexistence approach.
- Design integration patterns across batch, streaming, APIs, cloud services, identity, security, monitoring and downstream consumers.
- Develop migration principles covering data movement, ETL/ELT conversion, validation, reconciliation, cutover, rollback and post-migration stabilisation.
- Perform sizing, capacity, performance, resilience and cost considerations for the target platform.
- Produce architecture diagrams, solution design documents, capability mappings, migration roadmaps and technical decision records.
- Work with client architecture, security, data governance, infrastructure, application and delivery teams to obtain design alignment and approvals.
Required Skills & Experience
- Hands-on architecture experience with Databricks, Spark/PySpark, Delta Lake and modern lakehouse patterns.
- Strong knowledge of Cloudera/Hadoop ecosystems such as HDFS, Hive, Impala, Spark, Kafka and NiFi.
- Experience designing enterprise-scale data migration or data-platform modernisation programmes.
- Good understanding of data security, IAM, encryption, network controls, governance, lineage, auditability and regulatory requirements.
- Strong stakeholder management, workshop facilitation, documentation and presentation skills.
- Banking or financial-services delivery experience is preferred.
Preferred / Advantageous
- Databricks certifications or equivalent cloud/data architecture certifications.
- Experience with Azure Databricks, ADF, ADLS and enterprise CI/CD.
