Data Engineer - Creative Financial Staffing

Data Engineer

Hybrid| No sponsorship | Contract

Overview:

• Create and optimize ETL (Extract, Transform, Load) processes using AWS services such as Glue, S3, Lambda and Data Pipelines.
• Develop scalable data architectures that support our microservice-based data ecosystem.
• Utilize Terraform for infrastructure as code (IaC) to set up and manage resources.
• Implement best practices in data modeling, ensuring efficient data structures.
• Use PySpark and Python scripting for data transformation and enrichment.
• Work with Glue crawlers to discover and catalog data sources.
• Build and maintain Lambda functions for serverless data processing.
• Leverage Glue jobs for data transformation and orchestration.
• Utilize Glue catalog for metadata management.
• Implement logging, monitoring, and alerting using AWS CloudWatch and other relevant tools.
• Work with Delta Lake and Databricks for advanced analytics and data processing.
• Query data stored in S3 data lakes using AWS Athena.
• Interface with business stakeholders to gather requirements and deliver complete reporting solutions.
• Collaborate with cross-functional teams to ensure data consistency and accuracy.

Qualifications:

• Proficiency in Python and PySpark.
• Familiarity with Terraform for infrastructure provisioning.
• Experience with Glue jobs, Lambda functions, and Glue crawlers.
• Knowledge of Delta Lake, Databricks, and S3 data lake storage.

Benefits And Compensation:

• Compensation: $100,000 – $120,000
• Competitive PTO Package
• Retirement: 401(k) plan with a match
• Medical: Individual and Family plans available

#LI-KS4

#INMAY2024

#CBMAY2024