Databricks Administrator at LTIMindtree

About Us:

LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 700+ clients, LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes in a converging world. Powered by nearly 90,000 talented and entrepreneurial professionals across more than 30 countries, LTIMindtree — a Larsen & Toubro Group company — combines the industry-acclaimed strengths of erstwhile Larsen and Toubro Infotech and Mindtree in solving the most complex business challenges and delivering transformation at scale. For more information, please visit www.ltimindtree.com

Job Title: Databricks Administrator

Location: Charlotte NC

Job Summary:

We are seeking a skilled Databricks Administrator to manage and optimize our Databricks environment across cloud platforms such as Azure, AWS, or GCP. This role will focus on platform administration, performance tuning, security management, automation, and cost optimization while ensuring a highly available, scalable, and secure Databricks infrastructure.

The ideal candidate should have deep expertise in Databricks workspace management, cluster configurations, security controls, job scheduling, and integration with cloud data services like Azure Data Lake (ADLS), AWS S3, or Google Cloud Storage. Strong experience in Terraform, CI/CD, and monitoring tools is also preferred.

Key Responsibilities:

1. Databricks Platform Administration
2. Manage and maintain Databricks workspaces, clusters, and notebooks to ensure optimal performance and availability.
3. Set up and configure Databricks Runtime Versions, High-Concurrency Clusters, and Auto-scaling policies for efficient compute utilization.
4. Monitor cluster health, node utilization, and job execution performance for proactive troubleshooting.
5. Implement Databricks best practices to optimize query execution and data processing speeds.

1. Security, Access Control & Governance
2. Manage Databricks authentication and authorization (RBAC) using Azure AD, AWS IAM, or Google IAM.
3. Enforce fine-grained access controls (Table ACLs, Unity Catalog, SCIM integration) for secure data access.
4. Implement data encryption, tokenization, and masking strategies to comply with security policies.
5. Ensure GDPR, HIPAA, and other compliance requirements are met for data security and privacy.

1. Cost Optimization & Performance Tuning
2. Monitor Databricks cost usage and implement cost-control strategies (e.g., auto-termination, cluster rightsizing, spot instances).
3. Optimize Spark jobs, partitions, caching strategies, and parallel execution for faster performance.
4. Manage delta lake optimizations (Z-ordering, vacuum, compaction, and indexing) to improve query efficiency.
5. Perform profiling and debugging of slow-running queries and jobs.

1. Automation & Infrastructure as Code (IaC)
2. Automate Databricks workspace provisioning, cluster deployment, and job scheduling using Terraform or ARM templates.
3. Implement CI/CD pipelines for Databricks notebooks, workflows, and integrations using Azure DevOps, GitHub Actions, or Jenkins.
4. Develop APIs and scripts (Python, PowerShell, Bash) to automate Databricks administration tasks.
5. Maintain Infrastructure-as-Code (IaC) templates to ensure repeatable and scalable Databricks configurations.

1. Integration & Data Connectivity
2. Manage connectivity with external data sources, including Snowflake, Azure Data Lake (ADLS), AWS S3, Google BigQuery, SQL Server, and Oracle.
3. Configure Databricks JDBC/ODBC drivers, API integrations, and partner connectors.
4. Troubleshoot network, authentication, and API issues with cloud storage and external databases.
5. Ensure proper networking setup (VNET peering, private link, firewall rules) for secure and reliable connectivity.

1. Job Scheduling & Workflow Orchestration
2. Administer Databricks Job Scheduler and optimize notebook executions, streaming jobs, and workflows.
3. Integrate Databricks with Airflow, Control-M, or Azure Data Factory (ADF) for workflow automation.
4. Monitor long-running jobs, identify bottlenecks, and optimize execution timelines.

1. Monitoring & Troubleshooting
2. Set up Databricks monitoring dashboards using Azure Monitor, AWS CloudWatch, Datadog, or Prometheus.
3. Investigate job failures, performance degradation, and cluster errors using logs and metrics.
4. Ensure continuous monitoring of Databricks workspace utilization and incident management.

Required Skills & Experience:

Technical Skills:

? 5+ years of experience in Databricks administration on Azure, AWS, or GCP.

? Deep expertise in Apache Spark tuning, cluster scaling, and job optimization.

? Strong Databricks security administration experience (RBAC, Unity Catalog, SCIM, OAuth).

? Hands-on experience with Terraform, ARM Templates, or CloudFormation for Infrastructure-as-Code (IaC).

? Proficiency in Python, SQL, and Bash/PowerShell scripting for automation.

? Experience with monitoring & logging tools (Datadog, CloudWatch, Prometheus, Azure Monitor).

? Strong knowledge of cost optimization techniques for cloud-based Databricks deployments.

? Familiarity with Delta Lake, Lakehouse architecture, and MLflow integration.

? Experience integrating Databricks with Snowflake, Azure Synapse, AWS Redshift, or Google BigQuery.

? Working knowledge of DataOps, CI/CD pipelines, and DevOps tools (Azure DevOps, GitHub, Jenkins).

Soft Skills:

? Strong troubleshooting and problem-solving skills for critical system issues.

? Ability to work independently and collaborate across IT, security, and data engineering teams.

? Excellent communication skills for engaging with stakeholders and providing platform support.

? Strong documentation skills to maintain best practices, playbooks, and operational guides.

Benefits/perks listed below may vary depending on the nature of your employment with LTIMindtree (“LTIM”):

Benefits and Perks:

• Comprehensive Medical Plan Covering Medical, Dental, Vision
• Short Term and Long-Term Disability Coverage
• 401(k) Plan with Company match
• Life Insurance
• Vacation Time, Sick Leave, Paid Holidays
• Paid Paternity and Maternity Leave

The range displayed on each job posting reflects the minimum and maximum salary target for the position across all US locations. Within the range, individual pay is determined by work location and job level and additional factors including job-related skills, experience, and relevant education or training. Depending on the position offered, other forms of compensation may be provided as part of overall compensation like an annual performance-based bonus, sales incentive pay and other forms of bonus or variable compensation.

Disclaimer: The compensation and benefits information provided herein is accurate as of the date of this posting.

LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.

Safe return to office

In order to comply with LTIMindtree’ s company COVID-19 vaccine mandate, candidates must be able to provide proof of full vaccination against COVID-19 before or by the date of hire. Alternatively, one may submit a request for reasonable accommodation from LTIMindtree’s COVID-19 vaccination mandate for approval, in accordance with applicable state and federal law, by the date of hire. Any request is subject to review through LTIMindtree’s applicable processes.