Big Data Developer - Diverse Lynx

Big Data Developer (Spark & Scala) - Banking Domain (6-8 years experience)

Locations: Jersey City, NJ & Tampa, FL (Onsite)

About the Role:

We are seeking a highly skilled and experienced Big Data Developer to join our team in either Jersey City, NJ or Tampa, FL. In this role, you will play a critical role in developing and maintaining big data solutions using Apache Spark and Scala to unlock valuable insights from our banking domain data. You will collaborate with data engineers, data scientists, and business stakeholders to design, develop, and implement scalable data pipelines and applications.

Responsibilities:

• Develop and maintain big data applications using Spark and Scala for processing and analyzing large datasets related to banking operations.
• Design and implement scalable data pipelines using Spark to ingest, transform, and load data from various sources (e.g., relational databases, message queues, log files).
• Utilize Spark libraries (e.g., Spark SQL, MLlib) to perform data cleansing, transformation, feature engineering, and potentially machine learning tasks.
• Collaborate with data engineers to optimize Spark jobs for performance and efficiency.
• Work closely with data scientists to understand their analytical needs and translate them into production-ready Spark applications.
• Write clean, maintainable, and well-documented code adhering to best practices.
• Participate in code reviews and knowledge-sharing sessions with the team.
• Stay updated on the latest advancements in big data technologies (Spark, Scala) and the banking domain.

Qualifications:

Essential Skills:

• • 6-8 years of experience in software development with a focus on big data technologies.
  • Proven experience developing big data applications using Apache Spark and Scala.
  • Strong understanding of distributed computing frameworks and concepts (MapReduce, functional programming).
  • Experience with data ingestion techniques (e.g., Kafka, Flume) is a plus.
  • Familiarity with relational databases (e.g., MySQL, PostgreSQL) and SQL querying.
  • Experience working with data pipelines and data warehousing concepts.
  • Excellent problem-solving and analytical skills.
  • Ability to work independently and as part of a team.
  • Strong communication and collaboration skills.
  • Understanding of the banking domain (a plus).

Desirable Skills:

• • Experience with cloud platforms (AWS, Azure, GCP) for deploying Spark applications.
  • Knowledge of machine learning libraries (e.g., MLlib, TensorFlow) is a plus.
  • Experience with containerization technologies (Docker, Kubernetes).
  • Experience with DevOps tools and methodologies (CI/CD pipelines).
  • Experience working in an Agile development environment.