Cloud Infrastructure & Platform Specialist

About Our Client

Our client is a growing SaaS company providing cloud-based software that helps energy, utility, telecom, and infrastructure companies protect their critical network infrastructure. With nearly three decades of industry experience, the company serves customers across North America and continues to expand its platform with new data-driven and AI-powered capabilities.

About the Role

Our client is building a governed, multi-cloud data estate on Databricks to power cross-product insights. This role owns the cloud infrastructure and platform side that the Data Engineering, Data Science, MLOps, and Data Architecture teams depend on. You will provision and configure what these teams need in Databricks, in Azure (the backend for all Databricks solutions), and in AWS. Identity and Access Management is the core of the role, from IAM roles and access policies to the service principals that let other applications connect to Databricks. Requests often arrive through help desk tickets, but this is not a support position: your work is what allows the teams to move forward, faster and securely. The role is remote, based in India (Bangalore preferred, Pune also an option), Monday to Friday during standard business hours, with some overlap with the US team a few days a week.

Key Responsibilities

- Provision, configure, and manage the core cloud infrastructure behind a Databricks-based data estate, with Azure as the primary backend for Databricks and additional work in AWS.

- Make Azure configuration changes required for Databricks resources and settings to work as expected.

- Manage infrastructure components such as VPCs/VNets, subnets, route tables, network connectivity, S3 buckets and Azure Storage Accounts, compute resources, and private endpoints.

- Establish and maintain secure DEV, QA, and PROD environments with proper isolation and platform guardrails.

- Build, version, test, and deploy infrastructure changes through Infrastructure-as-Code, mainly Terraform (Azure ARM/Bicep and AWS CloudFormation also apply).

- Provision and manage IAM roles, service principals, managed identities, groups, and access policies across AWS, Azure, and Databricks.

- Create service principals so other applications can connect to Databricks and exchange results.

- Implement least-privilege access and role-based access control (RBAC), and support identity federation and automated identity lifecycle management (SSO, SCIM, SAML, OAuth).

- Review and process elevated or administrative access requests through controlled, auditable procedures, and take part in periodic access reviews.

- Create, configure, and administer Databricks workspaces, including Unity Catalog metastores, catalogs, schemas, storage credentials, external locations, and permissions.

- Define and maintain Databricks cluster policies, instance profiles, and storage credentials for secure, cost-controlled data access.

- Onboard Data Engineering, Data Science, MLOps, and Data Architecture projects by setting up the right catalogs, schemas, access, and infrastructure ahead of time.

- Resolve onboarding and access blockers such as missing admin privileges, cluster-policy conflicts, permission problems, storage-access failures, and authentication issues.

- Act as the primary escalation point for Databricks platform and infrastructure issues.

- Monitor security and governance across the platform, and flag and correct any security issue a team may introduce without realizing it.

- Implement secure private connectivity (AWS PrivateLink/VPC endpoints, Azure Private Link/private endpoints, VNet/VPC peering, network security groups, firewall controls, and egress restrictions).

- Implement encryption and key management using AWS KMS, AWS Secrets Manager, Azure Key Vault, or equivalent services.

- Maintain audit logging and provide evidence for access reviews, administrative activity, infrastructure changes, and security reviews.

- Enforce tagging standards, monitor cloud and Databricks spend, and apply policies that prevent unnecessary spend.

- Automate recurring work such as workspace provisioning, catalog and schema setup, IAM role assignment, and standard onboarding.

- Maintain clear runbooks, architecture documentation, access procedures, and onboarding guides.

Requirements

- 3-5 years of experience in cloud infrastructure, platform engineering, cloud administration, or a closely related role. Strong data engineers with solid Databricks experience and a genuine interest in moving into platform work are also considered.

- Strong, hands-on Identity and Access Management experience: IAM role provisioning, service principals, managed identities, RBAC, least-privilege access, SSO, SCIM, and SAML.

- Hands-on Databricks administration: workspace provisioning and configuration, Unity Catalog, metastores, catalogs and schemas, cluster policies, and data-access controls.

- Hands-on Microsoft Azure experience, as the backend for Databricks.

- Hands-on AWS experience, used alongside Azure.

- Working knowledge of Infrastructure-as-Code, mainly Terraform. Azure ARM/Bicep or AWS CloudFormation is also valued.

- Solid understanding of security and governance, with the judgment to spot and correct security issues across the platform.

- Familiarity with cloud networking and security fundamentals: VPC/VNet architecture, subnets and routing, private connectivity, network segmentation, secrets management, encryption, and key management.

- Ability to quickly troubleshoot infrastructure, networking, identity, permission, and platform-access issues.

- Strong communication skills, with the ability to explain infrastructure and access issues clearly to technical teams.

- Familiarity with enterprise compliance and security frameworks (SOC 2, ISO 27001, GDPR), audit logging, and access reviews.

- Based in India, with Bangalore preferred and Pune also an option, and able to overlap with US hours a few days a week.

Nice-to-Have

- Some data engineering background or experience.

- Databricks certification (e.g., Databricks Platform Administrator).

- Other cloud certifications, such as Azure Solutions Architect, Azure Administrator, or AWS Solutions Architect.

- Infrastructure and platform CI/CD experience, including GitHub Actions, Databricks Asset Bundles (DABs), and DEV to QA to PROD promotion.

- FinOps and cloud cost-management practices.

- Oil & gas, pipeline, utility, energy, or asset-integrity industry experience, including GIS or geospatial data.

What We Offer

- Be part of a dynamic and growing company that is well-respected in its industry.

- Competitive compensation based on experience and qualifications.

- Health insurance coverage.

AI Use in Hiring

As part of our hiring process, this role may use artificial intelligence or automated tools to assist with reviewing and screening applications. These tools support, but do not replace, human judgment in making hiring decisions.

Your application will only be counted once you complete the full registration process on the KeyStone platform, including creating your profile, uploading your CV, and submitting your application. The AI interview is optional and encouraged, but is not required for your application to be counted.