Big Data Engineer (Scala) - ASCENDING

Location: Remote, Anywhere within the US

Key Responsibilities:

• Design, build, and maintain scalable and high-performing data pipelines and architectures.
• Implement complex big data projects with a focus on collecting, parsing, managing, analyzing, and visualizing large sets of data to turn information into insights using Spark and SQL.
• Utilize AWS cloud services, primarily S3 and EMR, to build and maintain an efficient big data ecosystem.
• Develop and optimize ETL processes to integrate data from various sources, ensuring data quality and accessibility.
• Leverage coding skills in Scala to implement robust, scalable code that supports data processing and analytics applications.
• Work within the Hadoop ecosystem, utilizing tools such as Hive and Presto for data querying and manipulation.
• Collaborate with cross-functional teams to understand data needs and deliver solutions that meet business requirements.
• Ensure best practices in data management and security are followed within the big data infrastructure.

Required Qualifications:

• Minimum of 5 years of experience in big data engineering, with a proven track record of implementing large-scale data solutions.
• Expert knowledge of Spark and SQL, with extensive experience in data processing and analytics.
• Strong experience with AWS cloud services related to big data, including S3 and EMR.
• Proficient in coding with Scala, with the ability to develop high-quality, maintainable code.
• Solid understanding of the Hadoop ecosystem, including Hive and Presto.
• Experience in designing and developing ETL pipelines to support data transformation, loading, and extraction.
• Excellent problem-solving skills and the ability to work in a fast-paced environment.

Nice to Have:

• Experience with Athena and Lambda for data querying and serverless computing.
• Familiarity with other big data technologies and frameworks.
• Strong communication skills and the ability to work as part of a team or independently.