Senior Data Integrity Engineer at Sky Solutions LLC
HERNDON, VA 20171
About the Job
Position Title: Senior Data Integrity Engineer
Location: Remote (Preference for candidates in the DMV area)
About the Role:
As a Senior Data Integrity Engineer, you will join a dynamic data technology team that supports the development and implementation of enterprise level data products for business analytics and AI in the Federal government domain, specifically for CMS (Centers for Medicare and Medicad Services). You will be responsible for ensuring the data reliability, integrity, and efficiency of our data pipelines by utilizing a disciplined software development methodology and leading-edge technologies. You will also collaborate with cross-functional teams to understand data requirements and design scalable solutions that meet or exceed customer needs. You will display technical and functional competence in data management standards.
Responsibilities:
Design, implement, and maintain robust data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
Develop and implement data quality monitoring frameworks to detect anomalies, errors, and inconsistencies in the data pipeline.
Collaborate with cross-functional teams to understand data requirements and design scalable solutions to meet business needs.
Implement data governance policies and procedures to ensure compliance with regulatory requirements and industry best practices.
Optimize data pipeline performance, scalability, and reliability through automation, monitoring, and tuning.
Troubleshoot and resolve data pipeline issues in a timely manner to minimize disruptions to data workflows.
Research and evaluate emerging technologies and tools to continuously improve data pipeline architecture and efficiency.
Mentor junior team members and promote a culture of data reliability, quality, and innovation within the organization.
Continue to look for innovative and next generation solutions for solving data challenges.
Required Skills and Qualifications:
Bachelor's degree in computer science, Engineering, or related field.
Proven experience in designing, building, and optimizing data pipelines for analytics and AI applications.
Excellent problem-solving and troubleshooting skills with a focus on data quality and reliability.
Ability to adapt to changing business priorities and environments.
Significant experience interfacing with both customers and management across business and IT and track record of collaborating with IT, Business, and vendor teams to provide technical solutions and improvements, delivering end-to-end products/processes on schedule and budget, as per business requirements and SDLC standards.
Track record communicating issues & follow-up through issue resolution.
Provide Development & Production Support for applications.
Monitor production jobs, triage issues to appropriate team & resolve any issues in a timely manner.
Partner with other IT areas in resolving issues & improving processes.
Experience:
10+ Years of IT experience, with 8+ years of experience as a Senior level data reliability engineer or related data professional
5+ years of experience in modern data ingestion technology tools.
2+ years of experience in Collibra technology tool and data reliability engineering.
4+ years of experience and deeper knowledge in data engineering using Python, Pyspark and Databricks
Hands-on ETL tool Informatica PowerCenter, Informatica Cloud Data integration
Experience across multiple RDBMS and Managed databases.
7+ years of experience strong UNIX scripting.
Strong SQL, PL/SQL experience
Technical Proficiency:
Python
Pyspark
Databricks
Fivetran
Collibra
Informatica PowerCenter
Informatica Cloud Data integration
RDBMS and Managed databases
UNIX scripting
SQL
Preferred Qualifications:
Experience in working with Federal government clients, especially CMS (Centers for Medicare and Medicaid Services)
Familiarity with Big Data technologies like Hadoop and Cloud technologies
Soft Skills:
Self-starter who is fueled by collaboration, able to transform conceptual designs into reliable, scalable processes that meet or exceed customer needs.
Effective communication and presentation skills, both verbal and written.
Ability to work independently and as part of a team.
Ability to mentor and coach junior team members.
Ability to learn new technologies and tools quickly and adapt to changing requirements.
Working knowledge of the Finance or Insurance industry
PL/SQL
Location: Remote (Preference for candidates in the DMV area)
About the Role:
As a Senior Data Integrity Engineer, you will join a dynamic data technology team that supports the development and implementation of enterprise level data products for business analytics and AI in the Federal government domain, specifically for CMS (Centers for Medicare and Medicad Services). You will be responsible for ensuring the data reliability, integrity, and efficiency of our data pipelines by utilizing a disciplined software development methodology and leading-edge technologies. You will also collaborate with cross-functional teams to understand data requirements and design scalable solutions that meet or exceed customer needs. You will display technical and functional competence in data management standards.
Responsibilities:
Design, implement, and maintain robust data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
Develop and implement data quality monitoring frameworks to detect anomalies, errors, and inconsistencies in the data pipeline.
Collaborate with cross-functional teams to understand data requirements and design scalable solutions to meet business needs.
Implement data governance policies and procedures to ensure compliance with regulatory requirements and industry best practices.
Optimize data pipeline performance, scalability, and reliability through automation, monitoring, and tuning.
Troubleshoot and resolve data pipeline issues in a timely manner to minimize disruptions to data workflows.
Research and evaluate emerging technologies and tools to continuously improve data pipeline architecture and efficiency.
Mentor junior team members and promote a culture of data reliability, quality, and innovation within the organization.
Continue to look for innovative and next generation solutions for solving data challenges.
Required Skills and Qualifications:
Bachelor's degree in computer science, Engineering, or related field.
Proven experience in designing, building, and optimizing data pipelines for analytics and AI applications.
Excellent problem-solving and troubleshooting skills with a focus on data quality and reliability.
Ability to adapt to changing business priorities and environments.
Significant experience interfacing with both customers and management across business and IT and track record of collaborating with IT, Business, and vendor teams to provide technical solutions and improvements, delivering end-to-end products/processes on schedule and budget, as per business requirements and SDLC standards.
Track record communicating issues & follow-up through issue resolution.
Provide Development & Production Support for applications.
Monitor production jobs, triage issues to appropriate team & resolve any issues in a timely manner.
Partner with other IT areas in resolving issues & improving processes.
Experience:
10+ Years of IT experience, with 8+ years of experience as a Senior level data reliability engineer or related data professional
5+ years of experience in modern data ingestion technology tools.
2+ years of experience in Collibra technology tool and data reliability engineering.
4+ years of experience and deeper knowledge in data engineering using Python, Pyspark and Databricks
Hands-on ETL tool Informatica PowerCenter, Informatica Cloud Data integration
Experience across multiple RDBMS and Managed databases.
7+ years of experience strong UNIX scripting.
Strong SQL, PL/SQL experience
Technical Proficiency:
Python
Pyspark
Databricks
Fivetran
Collibra
Informatica PowerCenter
Informatica Cloud Data integration
RDBMS and Managed databases
UNIX scripting
SQL
Preferred Qualifications:
Experience in working with Federal government clients, especially CMS (Centers for Medicare and Medicaid Services)
Familiarity with Big Data technologies like Hadoop and Cloud technologies
Soft Skills:
Self-starter who is fueled by collaboration, able to transform conceptual designs into reliable, scalable processes that meet or exceed customer needs.
Effective communication and presentation skills, both verbal and written.
Ability to work independently and as part of a team.
Ability to mentor and coach junior team members.
Ability to learn new technologies and tools quickly and adapt to changing requirements.
Working knowledge of the Finance or Insurance industry
PL/SQL