Senior Data Engineer - Compunnel Inc.

Title : GCP Data Engineer

Location : Phoenix, AZ

Duration : Long Term

We are transforming data management and analytics to build cutting-edge solutions on the Google Cloud Platform (GCP). We are looking for an experienced and motivated GCP Data Engineer to join our growing team, who is passionate about migrating on-premise data systems to the cloud, and optimizing data pipelines. This role will work closely with data architects, analysts, and cloud engineers to build and migrate scalable data solutions on GCP.

Job Description

We are seeking a GCP Data Engineer with expertise in data migration from on-premise systems to Google Cloud Platform (GCP), especially utilizing BigQueryPySpark, and other modern cloud-based data technologies. The ideal candidate will have extensive experience in data engineering, cloud migration, and the ability to work on large-scale data transformations. Experience with Spring is a plus, as it will aid in building robust and scalable data applications.

Key Responsibilities

• Lead data migration initiatives from on-premise data storage and systems to GCP, ensuring high performance, scalability, and minimal downtime.
• Develop and optimize data pipelines leveraging BigQuery for large-scale analytics and querying of structured and unstructured data.
• Work with PySpark to design and implement distributed data processing tasks and data transformations.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions on GCP.
• Implement best practices for data governance, security, and compliance during migration and in cloud environments.
• Develop custom solutions using Spring Framework to integrate cloud services and manage data flows.
• Troubleshoot, optimize, and monitor data pipelines, ensuring seamless integration and high performance in GCP.
• Provide ongoing support and maintenance for cloud-based data systems.
• Document architecture, processes, and code for team knowledge-sharing and compliance purposes.

Required Qualifications

• Strong hands-on experience in migrating data from on-premise to cloud environments, particularly GCP.
• Proficiency with BigQuery for data analytics, including data modeling, query optimization, and performance tuning.
• Expertise in PySpark for building scalable data processing pipelines and working with large datasets.
• In-depth knowledge of cloud-based data architecture and engineering principles.
• Experience with cloud tools and services like Google Cloud StorageDataflowDataprocCloud Pub/Sub, etc.
• Strong programming skills in Python for data engineering tasks.
• Ability to design, develop, and optimize data pipelines for large datasets and high-volume workloads.
• Experience working with Spring or Spring Boot to develop backend applications is a plus.
• Excellent problem-solving skills, ability to debug complex data systems and optimize workflows.
• Strong communication skills and ability to collaborate with both technical and non-technical stakeholders.

Preferred Qualifications

• Google Cloud certifications (e.g., Associate Cloud EngineerProfessional Data Engineer) are a plus.
• Familiarity with data orchestration tools such as Airflow or Composer
• Experience with DockerKubernetes, or containerization technologies.
• Knowledge of SQL and working with relational databases.
• Exposure to DevOps practices for CI/CD pipelines and automation.

Why Join Us?

• Work in a collaborative, innovative, and dynamic environment.
• Opportunity to work with cutting-edge technologies on Google Cloud.
• Competitive salary and comprehensive benefits package.
• Career growth and development opportunities.
• Flexible work environment (remote and hybrid options available).

If you are passionate about GCP, cloud migration, data engineering, and building scalable solutions, we would love to hear from you!