Job Description
KEY ACCOUNTAIBILITIES:
- Provide technical expertise and leadership to design, development, implementation, testing, documentation key data components like streaming, pipelines and API services that support strategic business applicationsdata APIs that support mutual applications and data platforms integrations.
- Design real-time data service API components and services in API platform built within Group Data Platform (GODP).
- Lead optimization and enhancement of data streaming platform to achieve multi-BU multi-tenancy, scalability, maximum availability and stability under high load.
- Lead the design of database schema, query and aggregate data.
- Optimize the API infrastructure and ensure maximum availability and stability under high load.
- Develop new API features to serve web and mobile clients
- Build reusable code and libraries for future use.
- Ensure the technical feasibility of the delivered product. API
- Build clean and high-quality data processing or API codes to ensure that the code is well written and structured which allows the team to iterate quickly on projects.
- Consistently improve our development lifecycle by fine tuning the testing and provisioning process with methods such as infrastructure-as-code.
- Code-review to invest in spreading knowledge around the team, and keeping our code easy to read, test, and maintain.
- Presenting to leadership and stakeholders and supporting coordination of technical decisions.
QUALIFICATIONS / EXPERIENCE: University degree in computer science or related field.
6-8 years of hands-on experience in Python, Java/J2EE development.
Strong hands-on experience in data engineering, real-time data streaming (e.g. Kafka, Spark Streaming) and data API development, as well as data API integration development in an enterprise-level platform with number of system integration points to support complex operational and data manipulation processes. RESTful API and WSDL/SOAP web services are must-have
Strong knowledge of relational database, specifically MS SQL, MySQL and others RMDBS. Knowledge in CDC is a plus.
Familiar with prevalent ETL and data engineering tools like Databricks, Spark, Azure Data Factory, etc.
Familiar with Hadoop ecosystem.
Hands-on experience in Python, Scala, Java/J2EE development. Solid experience on .NET C# development would be bonus.
Well versed in security requirements for data applicationsAPI development, and able to apply the knowledge proactively in design, implementation and testing.
Experience documenting and testing API's using tools like Swagger, YAML, Postman, SoapUI, JMeter, and etc.
Prior knowledge/exposure to cloud computing (Azure or AWS) is a plus, mostly prefer Azure and Azure data components.
Technical expertise in softwarethe design, development, coding, testing, and debugging of software.
Experience with DevOps and CI/CD pipeline development.
Technical Skills:
In depth and solid experience in Python, Scala, SQL, Spark Streaming, Linux, Java/J2EEJave/J2EE, Java Spring, Javascript Express, Node.js, Python, .NET, C# and REST API.
Strong experience on database development and various database technologies such as MS SQL, Oracle, MySQL, Postgres, NoSQL, MongoDB and etc
Proven record on real-time data streaming and microservices development to handle low latency transactional processing
Prior experience working on stream processing systems such as enabling real time data analytics at scale using technologies similar to Apache Kafka, Flink, Spark, Druid, Cassandra, Elasticsearch etc.
Strong experience on managing API platform or product, such as KongAPI and MuleSoft.
Good Knowledge of major cloud platform such as Azure, AWS, GCP, or IBM. Azure is most preferable.
Big plus to have experience on data virtualization system such as Denodo.
Strong experience on container-based solution such as Azure Kubernetes (AKS) and Docker.