Lead Data Engineer at SpiceOrb
About the Job
Immediate Need Lead Data Engineer for Remote (Note: Fulltime Position)
Job Title: Lead Data Engineer
Location: Remote
Position: Fulltime
Job Description
Mentor a team of developers in building and maintaining high-quality, scalable, and secure data pipelines and analytics applications on the Databricks platform.
Work closely with data engineers, business stakeholders to understand their needs and design and develop solutions that meet those needs.
Design, develop, and test data pipelines and analytics applications using Databricks and other relevant technologies.
Responsibilities
Technical Leadership :
Lead and mentor a team of data engineers, providing guidance and support on technical aspects and best practices.
Define and implement the data architecture for the organization, ensuring scalability, performance, and security.
Evaluate and adopt new technologies and tools within the Databricks ecosystem.
Data Pipeline Development
Design and develop complex data pipelines for ingesting, processing, and transforming large datasets.
Ensure data quality and consistency throughout the data pipeline.
Optimize data pipelines for performance and resource efficiency.
Collaboration And Communication
Collaborate with cross-functional teams (business stakeholders, downstream user teams) to understand data needs and translate them into technical implementation.
Communicate effectively with stakeholders about data architecture, pipeline design, and operational challenges.
Document data pipelines and procedures for maintainability and knowledge sharing.
Problem-Solving And Innovation
Identify and troubleshoot issues within the data pipelines and infrastructure.
Proactively propose improvements and optimizations to the data platform.
Stay up to date on the latest trends and advancements in big data technologies.
Qualifications
7 + years of experience as a data engineer, with at least 2 years in a lead role.
Proven experience working with the Databricks platform.
Strong knowledge of big data technologies and frameworks (Spark, Kafka, etc.).
Experience with cloud platforms (AWS, Azure, GCP) is a plus.
Proficiency in programming languages like Python, Scala, or Java.
Excellent communication and collaboration skills.
Ability to work independently and as part of a team.
Regards
Thanks & Regards
Vijay Madi | Lead Recruitment
50 California Street, Suite 1500
San Francisco CA 94111
Direct +1 (669) 201-7883 | vijaym@spiceorb.com
www.spiceorb.com
Job Title: Lead Data Engineer
Location: Remote
Position: Fulltime
Job Description
Mentor a team of developers in building and maintaining high-quality, scalable, and secure data pipelines and analytics applications on the Databricks platform.
Work closely with data engineers, business stakeholders to understand their needs and design and develop solutions that meet those needs.
Design, develop, and test data pipelines and analytics applications using Databricks and other relevant technologies.
Responsibilities
Technical Leadership :
Lead and mentor a team of data engineers, providing guidance and support on technical aspects and best practices.
Define and implement the data architecture for the organization, ensuring scalability, performance, and security.
Evaluate and adopt new technologies and tools within the Databricks ecosystem.
Data Pipeline Development
Design and develop complex data pipelines for ingesting, processing, and transforming large datasets.
Ensure data quality and consistency throughout the data pipeline.
Optimize data pipelines for performance and resource efficiency.
Collaboration And Communication
Collaborate with cross-functional teams (business stakeholders, downstream user teams) to understand data needs and translate them into technical implementation.
Communicate effectively with stakeholders about data architecture, pipeline design, and operational challenges.
Document data pipelines and procedures for maintainability and knowledge sharing.
Problem-Solving And Innovation
Identify and troubleshoot issues within the data pipelines and infrastructure.
Proactively propose improvements and optimizations to the data platform.
Stay up to date on the latest trends and advancements in big data technologies.
Qualifications
7 + years of experience as a data engineer, with at least 2 years in a lead role.
Proven experience working with the Databricks platform.
Strong knowledge of big data technologies and frameworks (Spark, Kafka, etc.).
Experience with cloud platforms (AWS, Azure, GCP) is a plus.
Proficiency in programming languages like Python, Scala, or Java.
Excellent communication and collaboration skills.
Ability to work independently and as part of a team.
Regards
Thanks & Regards
Vijay Madi | Lead Recruitment
50 California Street, Suite 1500
San Francisco CA 94111
Direct +1 (669) 201-7883 | vijaym@spiceorb.com
www.spiceorb.com
Salary
125,000 - 150,000 /year