Data Engineer, Principal - Autodesk, Inc.
Novi, MI
About the Job
Employer Name: Autodesk, Inc.
Position: Data Engineer, Principal
Work location: Novi, MI
Wage: $138,200 - $180,000 / per year*
Job Duties: Design strategies for enterprise databases, data warehouse systems, and multidimensional networks. Set standards for database operations, programming, query processes, and security. Model, design, and construct large relational databases or data warehouses. Create and optimize data models for warehouse infrastructure and workflow. Integrate new systems with existing warehouse structure and refine system performance and functionality. Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, AWS 'big data' technologies and Snowflake. Build analytics tools that utilize the data pipeline to provide actionable insights into employee experience, operational efficiency, and other key business performance metrics. Work with stakeholders to assist with data-related technical issues and support associated data infrastructure needs. Build processes supporting data transformation, data structures, metadata, dependency, and workload management. Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS and Azure) and find ways to apply them back to help their team. Work with data scientists and analysts to strive for greater functionality in our data systems.
Some telecommuting is permitted.
Minimum Requirements: Bachelor's degree in Data/Computer Science, Information Technology, Information Systems, Engineering or related field and ten (10) years of progressive, post-baccalaureate experience in the job offered or in a data engineer-related occupation.
Some telecommuting is permitted.
Special Requirements:
Position requires at least nine (9) years of experience in each of the following skills:
1. Utilize knowledge of data modeling to design relational, star and snowflake schemas.
2. Utilize knowledge of ETL and ELT tools, including Abinitio, Informatica, Fivetran, Airbyte, and Airflow for data replication and change data capture to a destination system.
3. Utilize knowledge of big data technologies such as Hadoop, Apache Spark and Kafka, to process large datasets for both real-time and batch processing.
4. Utilize knowledge of relational databases (e.g., Teradata, SQL Server, MySQL, PostgreSQL), NoSQL databases (e.g., MongoDB, DynamoDB) and Cloud data warehouses (e.g., Snowflake, Amazon Redshift) to perform efficient data storage, retrieval, and management, ensuring robust data solutions.
5. Utilize knowledge of AWS services such as Lambda, EMR, Glue, IAM, Step Functions, S3, and Athena to design data pipelines for data capture and transformation.
6. Utilize knowledge of programming languages such as Python, Java, PySpark, Spark, and SQL to develop scripts for data pipelines and handle tasks including data cleaning, processing, and transformation.
7. Utilize knowledge of Terraform and AWS Cloud Formation to deploy data pipelines in cloud platforms.
Position requires at least one (1) year of experience in each of the following skills:
8. Utilize knowledge of Amazon API Gateway/Swagger and programming languages such as Python and Java to develop RESTful API endpoints to enable real-time queries through web services.
9. Implement data protection measures such as pseudonymization, encryption, versioning, access control list (ACL) to comply with data privacy regulations like GDPR and
Position: Data Engineer, Principal
Work location: Novi, MI
Wage: $138,200 - $180,000 / per year*
Job Duties: Design strategies for enterprise databases, data warehouse systems, and multidimensional networks. Set standards for database operations, programming, query processes, and security. Model, design, and construct large relational databases or data warehouses. Create and optimize data models for warehouse infrastructure and workflow. Integrate new systems with existing warehouse structure and refine system performance and functionality. Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, AWS 'big data' technologies and Snowflake. Build analytics tools that utilize the data pipeline to provide actionable insights into employee experience, operational efficiency, and other key business performance metrics. Work with stakeholders to assist with data-related technical issues and support associated data infrastructure needs. Build processes supporting data transformation, data structures, metadata, dependency, and workload management. Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS and Azure) and find ways to apply them back to help their team. Work with data scientists and analysts to strive for greater functionality in our data systems.
Some telecommuting is permitted.
Minimum Requirements: Bachelor's degree in Data/Computer Science, Information Technology, Information Systems, Engineering or related field and ten (10) years of progressive, post-baccalaureate experience in the job offered or in a data engineer-related occupation.
Some telecommuting is permitted.
Special Requirements:
Position requires at least nine (9) years of experience in each of the following skills:
1. Utilize knowledge of data modeling to design relational, star and snowflake schemas.
2. Utilize knowledge of ETL and ELT tools, including Abinitio, Informatica, Fivetran, Airbyte, and Airflow for data replication and change data capture to a destination system.
3. Utilize knowledge of big data technologies such as Hadoop, Apache Spark and Kafka, to process large datasets for both real-time and batch processing.
4. Utilize knowledge of relational databases (e.g., Teradata, SQL Server, MySQL, PostgreSQL), NoSQL databases (e.g., MongoDB, DynamoDB) and Cloud data warehouses (e.g., Snowflake, Amazon Redshift) to perform efficient data storage, retrieval, and management, ensuring robust data solutions.
5. Utilize knowledge of AWS services such as Lambda, EMR, Glue, IAM, Step Functions, S3, and Athena to design data pipelines for data capture and transformation.
6. Utilize knowledge of programming languages such as Python, Java, PySpark, Spark, and SQL to develop scripts for data pipelines and handle tasks including data cleaning, processing, and transformation.
7. Utilize knowledge of Terraform and AWS Cloud Formation to deploy data pipelines in cloud platforms.
Position requires at least one (1) year of experience in each of the following skills:
8. Utilize knowledge of Amazon API Gateway/Swagger and programming languages such as Python and Java to develop RESTful API endpoints to enable real-time queries through web services.
9. Implement data protection measures such as pseudonymization, encryption, versioning, access control list (ACL) to comply with data privacy regulations like GDPR and
Source : Autodesk, Inc.