Senior Software Engineer (PySpark)

We are seeking a highly skilled Senior Software Engineer with expertise in PySpark to join our dynamic development team. The ideal candidate will have a strong background in software development, a passion for data processing, and the ability to work collaboratively in a fast paced environment. This role is critical in driving our data engineering initiatives and ensuring the successful delivery of high quality software solutions.

Responsibilities:

• Design, develop, and maintain scalable data processing applications using PySpark.

• Collaborate with cross functional teams to gather requirements and translate them into technical specifications.

• Implement data pipelines and ETL processes to support data analytics and reporting.

• Optimize existing data processing workflows for performance and efficiency.

• Conduct code reviews and provide mentorship to junior developers.

• Participate in Agile development processes, including sprint planning and retrospectives.

• Stay updated with the latest industry trends and technologies related to data engineering and software development.

Mandatory Skills:

• Proven experience in software development with a strong focus on PySpark.

• Solid understanding of data structures, algorithms, and software design principles.

• Experience with big data technologies and frameworks such as Hadoop and Spark.

• Proficiency in programming languages such as Python and Java.

• Strong problem solving skills and the ability to work independently.

• Excellent communication and collaboration skills.

Preferred Skills:

• Experience with cloud platforms such as AWS, Azure, or Google Cloud.

• Familiarity with data visualization tools like Tableau or Power BI.

• Knowledge of database technologies such as SQL, NoSQL, and data warehousing concepts.

• Experience with containerization technologies like Docker and orchestration tools like Kubernetes.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.

• 5 -7 years of professional experience in software development, with a focus on data engineering.

• Strong portfolio of projects demonstrating expertise in PySpark and data processing