Data Engineer - Onsite at Viva USA Inc.
Dallas, TX 75201
About the Job
Title: Data Engineer - Onsite
Mandatory skills:
Data Engineering
Databricks, data modeling, warehousing, pipelines, data engineering
data pipelines, data integration, data systems, data science, data solution, data infrastructure
data integrity, data consistency, data quality, data workflow, data anomalies
ETL process, machine learning infrastructure
SQL, Python, Databricks ETL, SQL Server
Teradata SQL, MS SQL, IBM DB2
telecoms, mobile providers, ISP, cable, technical support
Description:
Data Engineer
The client is looking for a Data Engineer to join our Advanced Analytics team under the Wholesale organization. This engineer will design and deliver data solutions for the analytics and data science teams in the consumer organization. The engineer will be able to work with all types of interesting data: fiber network and outage data, billing, digital behavior, customer interactions, and customer demographic data.
In this role, you will design and build out the modern data infrastructure to streamline the data science workflows for the data science and overarching advanced analytics teams in the consumer organization. These teams are primarily responsible for building predictive models and driving retention and sales strategy for our consumer base.
The manager is looking for extensive knowledge in Databricks platform. As we migrate our existing processes from SQL Server to Databricks I need someone that can assist in pipeline builds, curated data sets, and communicate with our IT department on needs.
Responsibilities:
Develop and implement ETL pipelines to efficiently transfer data from SQL Server to Databricks, ensuring accuracy, scalability, and optimal performance.
Validate and transform data within Databricks, including testing for data integrity, consistency, and quality across ingestion and processing stages.
Collaborate with cross-functional teams to design, build, and optimize curated tables in Databricks for analytics and reporting purposes.
Monitor and troubleshoot data workflows, addressing any issues related to performance, reliability, or data anomalies.
Document data engineering processes and provide technical support for ongoing maintenance and enhancements to the Databricks environment.
Required Qualifications:
2+ years of experience as a Data Engineer in a similar role
Experience with data modeling, warehousing and building pipelines
Proven experience in designing and implementing comprehensive data pipelines for a variety of flows (data integration across systems, ETL processes, machine learning infrastructures)
Proficient in SQL and Python
Experience in Databricks ETL
Preferred Qualifications:
The more experience, the better, when it comes to the AWS ecosystem (e.g. GLUE, Athena, S3, Lambda, IAM, SageMaker, CloudWatch, API Gateway), Delta Lake, PySpark, Apache Spark, Airflow, APIs (REST, SOAP, RPC), streaming event data
Understanding of system architecture and experience with large distributed systems (familiarity with the Apache Spark ecosystem)
Has prior experience working with Teradata SQL, MS SQL, and IBM DB2 or similar dialects
Has prior experience in continuous integration and continuous deployment of large scalable data systems
Has prior experience working with and supporting a data science team
Familiarity with working on data involving telecoms, mobile providers, ISPs or cable companies
Note :
Onsite
VIVA USA is an equal opportunity employer and is committed to maintaining a professional working environment that is free from discrimination and unlawful harassment. The Management, contractors, and staff of VIVA USA shall respect others without regard to race, sex, religion, age, color, creed, national or ethnic origin, physical, mental or sensory disability, marital status, sexual orientation, or status as a Vietnam-era, recently separated veteran, Active war time or campaign badge veteran, Armed forces service medal veteran, or disabled veteran. Please contact us at hr@viva-it.com for any complaints, comments and suggestions.
Contact Details :
Account co-ordinator: Amirtha Valli Ravanan, Phone No: (847) 461-3596, Email: staffing10@viva-it.com
VIVA USA INC.
3601 Algonquin Road, Suite 425
Rolling Meadows, IL 60008
staffing10@viva-it.com | http://www.viva-it.com
Mandatory skills:
Data Engineering
Databricks, data modeling, warehousing, pipelines, data engineering
data pipelines, data integration, data systems, data science, data solution, data infrastructure
data integrity, data consistency, data quality, data workflow, data anomalies
ETL process, machine learning infrastructure
SQL, Python, Databricks ETL, SQL Server
Teradata SQL, MS SQL, IBM DB2
telecoms, mobile providers, ISP, cable, technical support
Description:
Data Engineer
The client is looking for a Data Engineer to join our Advanced Analytics team under the Wholesale organization. This engineer will design and deliver data solutions for the analytics and data science teams in the consumer organization. The engineer will be able to work with all types of interesting data: fiber network and outage data, billing, digital behavior, customer interactions, and customer demographic data.
In this role, you will design and build out the modern data infrastructure to streamline the data science workflows for the data science and overarching advanced analytics teams in the consumer organization. These teams are primarily responsible for building predictive models and driving retention and sales strategy for our consumer base.
The manager is looking for extensive knowledge in Databricks platform. As we migrate our existing processes from SQL Server to Databricks I need someone that can assist in pipeline builds, curated data sets, and communicate with our IT department on needs.
Responsibilities:
Develop and implement ETL pipelines to efficiently transfer data from SQL Server to Databricks, ensuring accuracy, scalability, and optimal performance.
Validate and transform data within Databricks, including testing for data integrity, consistency, and quality across ingestion and processing stages.
Collaborate with cross-functional teams to design, build, and optimize curated tables in Databricks for analytics and reporting purposes.
Monitor and troubleshoot data workflows, addressing any issues related to performance, reliability, or data anomalies.
Document data engineering processes and provide technical support for ongoing maintenance and enhancements to the Databricks environment.
Required Qualifications:
2+ years of experience as a Data Engineer in a similar role
Experience with data modeling, warehousing and building pipelines
Proven experience in designing and implementing comprehensive data pipelines for a variety of flows (data integration across systems, ETL processes, machine learning infrastructures)
Proficient in SQL and Python
Experience in Databricks ETL
Preferred Qualifications:
The more experience, the better, when it comes to the AWS ecosystem (e.g. GLUE, Athena, S3, Lambda, IAM, SageMaker, CloudWatch, API Gateway), Delta Lake, PySpark, Apache Spark, Airflow, APIs (REST, SOAP, RPC), streaming event data
Understanding of system architecture and experience with large distributed systems (familiarity with the Apache Spark ecosystem)
Has prior experience working with Teradata SQL, MS SQL, and IBM DB2 or similar dialects
Has prior experience in continuous integration and continuous deployment of large scalable data systems
Has prior experience working with and supporting a data science team
Familiarity with working on data involving telecoms, mobile providers, ISPs or cable companies
Note :
Onsite
VIVA USA is an equal opportunity employer and is committed to maintaining a professional working environment that is free from discrimination and unlawful harassment. The Management, contractors, and staff of VIVA USA shall respect others without regard to race, sex, religion, age, color, creed, national or ethnic origin, physical, mental or sensory disability, marital status, sexual orientation, or status as a Vietnam-era, recently separated veteran, Active war time or campaign badge veteran, Armed forces service medal veteran, or disabled veteran. Please contact us at hr@viva-it.com for any complaints, comments and suggestions.
Contact Details :
Account co-ordinator: Amirtha Valli Ravanan, Phone No: (847) 461-3596, Email: staffing10@viva-it.com
VIVA USA INC.
3601 Algonquin Road, Suite 425
Rolling Meadows, IL 60008
staffing10@viva-it.com | http://www.viva-it.com