Sr Data Engineer - eTek IT Services, Inc.

Overview

The Senior Data Engineer plays a crucial role in building and maintaining the data infrastructure essential for driving decision-making processes in the organization. This position is vital in ensuring that data sources are robust, reliable, and scalable to support complex analytical processes. The Senior Data Engineer collaborates with data analysts, data scientists, and other stakeholders to understand data needs and translate them into technical requirements. With a strong focus on optimizing data flow, improving quality, and ensuring secure access, this role directly contributes to enhancing data-driven insights and operational efficiency. Located in Minnesota, this position demands a proactive individual who is well-versed in data engineering practices and emerging technologies, capable of managing large datasets and ensuring seamless integration across various platforms.

Key Responsibilities

Design and implement scalable data pipelines to process large volumes of data.

Develop, test, and maintain ETL processes to ensure timely data availability.

Manage and optimize data storage solutions, including data warehouses and lakes.

Work collaboratively with data scientists to enable advanced analytics.

Perform data modeling and schema design to facilitate efficient data retrieval.

Ensure data quality and integrity through validation and monitoring processes.

Leverage cloud technologies such as AWS, Azure, or GCP for data solutions.

Create and maintain documentation related to data processes and architecture.

Participate in code reviews to promote best practices within the engineering team.

Implement security measures to protect sensitive data and comply with regulatory standards.

Analyze and resolve performance issues in data processing workflows.

Stay current with industry trends and emerging technologies in data engineering.

Assist in mentoring junior data engineers in technical skills and best practices.

Communicate effectively with stakeholders to gather requirements and provide updates.

Conduct impact analysis of new data initiatives and propose optimization strategies.

Required Qualifications

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Minimum of 5 years of experience in data engineering or a similar role.

Proficiency in SQL and experience with relational databases like PostgreSQL, MySQL.

Strong programming skills in Python; familiarity with Java or Scala is a plus.

Experience with big data technologies such as Hadoop, Spark, or Kafka.

Solid understanding of data warehousing concepts and practices.

Hands-on experience with cloud data solutions (AWS, Azure, GCP).

Knowledge of data quality and governance best practices.

Ability to work in a fast-paced, collaborative team environment.

Strong analytical and problem-solving skills.

Familiarity with version control systems like Git.

Excellent communication skills, both verbal and written.

Aptitude for mentoring and guiding less experienced team members.

Experience in Agile methodologies is advantageous.

Ability to adapt to changing business needs and priorities.

Certification in relevant data engineering tools and platforms is a bonus.