AI(Gen)/Data Lake Engineer - Collins Consulting Inc
Mettawa, IL 60045
About the Job
This is a onsite position.
Reporting to the Lead Technical Architect for our Commercial Data Lake for Finance (CDL-F), the Engineer will be responsible for the technical development of data, analytics, and GenAI solutions in support of the US Therapeutics Finance business unit. They will assist in the architecture, design, build, and testing of these applications, ensuring that these fit-for-purpose solutions meet current and future business needs. The Engineer will also troubleshoot system issues including the performance of root cause analysis and recommendations for remediating these issues.
Job Description
Responsibilities
" Independently design and develop data, software, or technology solutions to answer business questions. Demonstrate proficiency across a range of technologies related to programming languages, data integration, data warehousing, and data visualization and analysis.
" Play a key role in creating and maintain optimal data pipeline architecture
" Assemble large, complex data sets that meet functional / non-functional business requirements.
" Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability & elasticity etc.
" Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL/NoSQL and AWS big data technologies.
" Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
" Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
" Continuously seek to improve existing methods and processes.
Qualifications
" Bachelors degree in relevant field and at least 4 years relevant experience OR Masters degree in relevant field and at least 2 years experience.
" Hands-on experience with big data tools: Hadoop, Spark, Scala, HDFS - EMR, Cloudera etc
" Hands-on experience with AWS cloud services as a platform and as an infrastructure
" Working experience on any of the reporting tools such as Power BI, Qlik, etc
" Implementation experience of DevOps CICD automations, Docker, Kubernetes is preferred
" Good communication and interpersonal skills, with proven ability to build trust.
" Interest in technology trends and application of technology to improve end user s experience
" Ability to understand the broad objectives of a project as well as his/her role in achieving those objectives and modify approach when required. Escalates open issues to management to ensure execution success and keeps stakeholders informed through effective verbal and written communication.
" Learn, understand and master new methods and technologies, and act as a resource or mentor for others.
" Partner with vendors and provide input into overall project time and cost estimates.
" Produces and maintains required technical documentation associated with work product.
" Understand and adhere to corporate standards regarding applicable Corporate and Divisional Policies, including code of conduct, safety, SOX compliance, data security, and the software development lifecycle.
Preferred:
" Strong SQL skills with the ability to write efficient queries
" Experience analyzing large sets of data from various sources
" Ability to troubleshoot issues and provide timely resolution for end-users
" Ability to work independently and as part of a team in a fast-paced environment
" Understanding of Data Governance concepts and principles is a plus
" Any experience with Data Lineage tools like Alation is a plus
" Experience with designing or implementing GenAI solutions
Reporting to the Lead Technical Architect for our Commercial Data Lake for Finance (CDL-F), the Engineer will be responsible for the technical development of data, analytics, and GenAI solutions in support of the US Therapeutics Finance business unit. They will assist in the architecture, design, build, and testing of these applications, ensuring that these fit-for-purpose solutions meet current and future business needs. The Engineer will also troubleshoot system issues including the performance of root cause analysis and recommendations for remediating these issues.
Job Description
Responsibilities
" Independently design and develop data, software, or technology solutions to answer business questions. Demonstrate proficiency across a range of technologies related to programming languages, data integration, data warehousing, and data visualization and analysis.
" Play a key role in creating and maintain optimal data pipeline architecture
" Assemble large, complex data sets that meet functional / non-functional business requirements.
" Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability & elasticity etc.
" Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL/NoSQL and AWS big data technologies.
" Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
" Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
" Continuously seek to improve existing methods and processes.
Qualifications
" Bachelors degree in relevant field and at least 4 years relevant experience OR Masters degree in relevant field and at least 2 years experience.
" Hands-on experience with big data tools: Hadoop, Spark, Scala, HDFS - EMR, Cloudera etc
" Hands-on experience with AWS cloud services as a platform and as an infrastructure
" Working experience on any of the reporting tools such as Power BI, Qlik, etc
" Implementation experience of DevOps CICD automations, Docker, Kubernetes is preferred
" Good communication and interpersonal skills, with proven ability to build trust.
" Interest in technology trends and application of technology to improve end user s experience
" Ability to understand the broad objectives of a project as well as his/her role in achieving those objectives and modify approach when required. Escalates open issues to management to ensure execution success and keeps stakeholders informed through effective verbal and written communication.
" Learn, understand and master new methods and technologies, and act as a resource or mentor for others.
" Partner with vendors and provide input into overall project time and cost estimates.
" Produces and maintains required technical documentation associated with work product.
" Understand and adhere to corporate standards regarding applicable Corporate and Divisional Policies, including code of conduct, safety, SOX compliance, data security, and the software development lifecycle.
Preferred:
" Strong SQL skills with the ability to write efficient queries
" Experience analyzing large sets of data from various sources
" Ability to troubleshoot issues and provide timely resolution for end-users
" Ability to work independently and as part of a team in a fast-paced environment
" Understanding of Data Governance concepts and principles is a plus
" Any experience with Data Lineage tools like Alation is a plus
" Experience with designing or implementing GenAI solutions
Source : Collins Consulting Inc