Sr Data Engineer at S&S Tech Solutions LLC

S & S Tech solutions  is looking for a Senior Data Engineer to join our Data Engineering Team and build world-class data solutions and applications that power crucial business decisions throughout the organization. If you are a seasoned Senior Big Data Software Engineer looking for a challenging and rewarding opportunity, we invite you to apply and become an integral part of our dynamic team. As a Senior Engineer, you will play a crucial role in designing and implementing Big Data solutions for our diverse range of projects. The ideal candidate should have extensive experience in Big Data and related technologies, with a focus on Apache Spark, Java or Python, and AWS.

Role and Responsibilities

• Conduct requirements gathering and project scoping sessions with subject matter experts, business users, and executive stakeholders to discover and define business data needs.
• Design and implement end-to-end Big Data solutions for complex business requirements.
• Collaborate with cross-functional teams to understand project requirements and deliver high-quality software solutions.
• Support client technical architects in conducting PoCs, including cost evaluations, of various services and other tools
• Work with analysts to scale value-creating capabilities, including data integrations and transformations
• Oversee testing
• Drive data reliability, data integrity, and data governance, enabling accurate, consistent, and trustworthy data sets, business intelligence products, and analyses. Ensure data quality through building data quality frameworks
• Oversee incident management & request fulfilment
• Ensure SLAs are met, as per documented SLA definitions
• Manage escalations and approvals
• Stay current with industry trends and advancements in Big Data technologies, contributing to the continuous improvement of our development processes.

Qualifications

Minimum Qualifications

• 10+ years of professional experience as a Data Engineer or in a similar role.
• Strong programming skills using Python or Java.
• Bachelor’s degree in business information systems (IS), computer science or related field, or equivalent-related IT experience.
• Experience building real-time data pipelines using a micro-services architecture.
• Experience working with AWS data services or similar Apache projects (Spark, Flink, Hive, and Kafka).
• Working knowledge of relational databases and query authoring via SQL.
• Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team.
• Have practical experience in Python, PySpark, SQL
• Strong proficiency with Hadoop and Hive.
• Well-versed in modern software development practices (Agile, TDD, CICD).
• Good Communication skill
• Able to interact with clients independently

Preferred Qualifications/skills

• Experience working with native Cloud data services, specifically AWS.
• Experience operating large data warehouses
• AWS certified Cloud Data Engineer
• Databricks certified Data Engineer