Infrastructure Engineer (Databricks) - Open Systems Technologies
Bloomfield, CT 06152
About the Job
Infrastructure Engineer (Data Bricks)
Duties:
Provides counsel and advice to top management on significant Infrastructure matters, often requiring coordination between organizations. Serves as the lead technical consultant responsible for design, engineering and development of information technology solutions. Leads the design, analysis, development and implementation of engineering infrastructure solutions of projects and/or work requests for complex business solutions. Troubleshoots and proceeds on resolving complex infrastructure compatibility issues based on system architecture, project requirements, customer expectations, etc. May be involved in the creation of design specifications and the development and application of utility programs and may resolve escalated infrastructure compatibility problems as they arise. Responsibilities may include risks assessments, partnering on capacity planning, budget forecasting, and the diagnosis and correction of application/network problems. May serve as the primary contact for vendors. Technologies span mainframe, midrange, desktop, server, network data, voice, messaging, web, database, and output/receiving. Focuses on providing thought leadership and technical expertise across multiple disciplines.
Recognized internally as “the go-to person” for the most complex Infrastructure Engineering assignments.
Top skills:
Duties:
Provides counsel and advice to top management on significant Infrastructure matters, often requiring coordination between organizations. Serves as the lead technical consultant responsible for design, engineering and development of information technology solutions. Leads the design, analysis, development and implementation of engineering infrastructure solutions of projects and/or work requests for complex business solutions. Troubleshoots and proceeds on resolving complex infrastructure compatibility issues based on system architecture, project requirements, customer expectations, etc. May be involved in the creation of design specifications and the development and application of utility programs and may resolve escalated infrastructure compatibility problems as they arise. Responsibilities may include risks assessments, partnering on capacity planning, budget forecasting, and the diagnosis and correction of application/network problems. May serve as the primary contact for vendors. Technologies span mainframe, midrange, desktop, server, network data, voice, messaging, web, database, and output/receiving. Focuses on providing thought leadership and technical expertise across multiple disciplines.
Recognized internally as “the go-to person” for the most complex Infrastructure Engineering assignments.
- Install, configure, and maintain Databricks clusters and workspaces.
- Monitor cluster performance, resource utilization, and troubleshoot issues to ensure optimal performance.
- Implement and manage access controls and security policies to protect sensitive data.
- Provide technical support to Databricks users, including troubleshooting and issue resolution.
- Conduct training sessions to educate users on platform best practices and capabilities.
- Optimize cluster configurations for performance and cost-effectiveness.
- Monitor and manage auto-scaling to ensure resources are allocated efficiently.
- Implement and enforce security policies, access controls, and encryption mechanisms.
- Stay up-to-date with security best practices and compliance requirements.
- Develop and maintain backup and disaster recovery strategies to ensure data integrity and availability.
- Collaborate with data engineers to integrate Databricks with other data sources, data warehouses, and data lakes.
- Monitor and manage platform costs, including optimizing resource utilization to control expenses.
- Maintain detailed documentation of configurations, procedures, and best practices.
- Collaborate with cross-functional teams, including data scientists, data engineers, and business analysts,
- To understand their requirements and provide technical solutions
- Hands-on experience in GitHub and Terraform
- will be responsible for building and maintaining CI/CD workflows, managing Infrastructure as code(IaC) using Terraform and ensuring seamless collaboration across development and operations teams.
- Experience with Splunk and SNOW is a valuable plus
Top skills:
- Databricks
- Snowflake
- Devops
- Python
- git
- Terraform
Source : Open Systems Technologies