Data Engineer - American Technology Consulting

REMOTEURBANDALE, IAPrimary Roles• Create and update API interfaces to MySQL backend databases. TheAPI is hosted on AWS ECS and written using Python.• Ingest data into the Enterprise Data Lake from various database/file sources• Transform data and create common tables using Pyspark and SQL• Load data into MySQL from the Enterprise Data Lake• Analyze data using SQL and Pyspark• Create and maintain updates to AWS resourcesSupporting Roles• Work closely with the front end developer on the API specification• Build test scripts in Python to test data replication processes• Work closely with the product manager and end users to understand requirementsRequired SkillsExperience requirements - at least 3 - 5 years of experience withSQL, Python and cloud computing (AWS preferred)Languages and Frameworks: Python, SQL, PysparkCore understanding of AWS services including IAM, Lambda, S3, RDS, ECS and EC2Experience developing Databricks jobs and pipelinesOther: Docker, containerization concepts, REST API concepts (calling REST APIsusing Python)Additional SkillsTerraform, Github Actions, Datadog, AWS Database Migration Service, ApacheAirflow, REST API development in PythonThis position is available for remote work