Data Lead/Architect - Expert In Recruitment Solutions

ISelin, NJ 08830

About the Job

Position Overview:
We are seeking a skilled and motivated Lead/Architect Engineer with expertise in Spark SQL, T-SQL, and a strong background in the Azure ecosystem. This role focuses on designing, building, and optimizing robust data engineering solutions to support data analytics and business intelligence initiatives. The ideal candidate is passionate about building efficient and scalable data systems, thrives in a collaborative environment, and has hands-on experience with modern data engineering tools and cloud platforms.

Key Responsibilities:

Data Engineering:
- Design, develop, and maintain ETL/ELT pipelines using Spark SQL, T-SQL, and other relevant tools.
- Implement data integration and transformation workflows for structured and unstructured data sources.
- Optimize data pipelines for performance, reliability, and scalability.
Azure Ecosystem Expertise:
- Utilize Azure services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, and Azure Blob Storage for data movement, transformation, and storage.
- Leverage Azure SQL Database and related services to build efficient and scalable data solutions.
- Explore and implement best practices for data management and orchestration in the Azure cloud.
Data Modeling and Development:
- Design and implement data models to support business intelligence, analytics, and reporting.
- Collaborate with stakeholders to understand business requirements and translate them into technical solutions.
- Develop reusable data assets such as scripts, templates, and frameworks for engineering workflows.
Collaboration:
- Work closely with data analysts, data scientists, and business teams to support their data needs.
- Collaborate with software engineers and architects to integrate data solutions into broader systems.
Performance and Optimization:
- Conduct performance tuning for Spark SQL and T-SQL queries and workflows.
- Optimize data pipelines and data structures for faster processing and minimal latency.
Innovation and Best Practices:
- Stay updated on emerging technologies and trends in data engineering and cloud platforms.
- Contribute to the continuous improvement of data engineering processes and frameworks.

Required Qualifications:

Bachelor's degree in Computer Science, Data Engineering, or a related field (or equivalent practical experience).
3+ years of experience with Spark SQL and T-SQL in a professional setting.
Strong experience in building ETL/ELT pipelines and data integration workflows.
Expertise with Azure cloud services, including Azure Data Factory, Azure Synapse Analytics, Azure Blob Storage, and Azure Data Lake.
Proficient in designing and optimizing complex data pipelines for both batch and streaming data.
Strong understanding of data warehousing concepts, dimensional modeling, and big data processing.

Preferred Qualifications:

Experience with CI/CD workflows using Azure DevOps or similar tools.
Familiarity with programming languages like Python or Scala for data engineering tasks.
Knowledge of tools and platforms like Databricks or Apache Hadoop.
Experience working in an Agile development environment.
Azure certifications such as Azure Data Engineer Associate or similar are a plus.

Soft Skills:

Strong problem-solving and analytical skills.
Excellent communication and collaboration skills to work effectively with technical and non-technical teams.
A proactive attitude toward learning new tools and technologies.
Ability to manage multiple priorities and meet deadlines in a fast-paced environment.

Source : Expert In Recruitment Solutions