Data Engineer - virtusa

Title: Data Engineer

Minimum 6 years of experience in data engineering. 
Extensive experience with Apache Spark and cloud-based data processing.

Skills and Technologies:

Proficiency in Java/Scala/Python for Spark programming.
Experience with cloud platforms (e.g., AWS, Azure, GCP).
Expertise in big data technologies (e.g., Hadoop, Hive, Kafka) and a proven ability to design, implement, and optimize end-to-end data pipelines for efficient data flow.
Familiarity with Docker and Kubernetes is a plus.
Strong understanding of relational and NoSQL databases, with the ability to design and optimize data storage solutions.

Responsibilities:

Design, develop, and optimize data processing systems.
Implement and maintain data pipelines in a cloud environment.
Collaborate with data architects for infrastructure support.
Troubleshoot and optimize Spark and cloud-based processes.
Ensure adherence to data governance and compliance standards, implementing best practices for data quality and security.