Lead the design, development, and implementation of scalable data solutions using Databricks on cloud platforms. Architect data pipelines, optimize performance, and ensure data governance across enterprise analytics initiatives.

About the Company (InfoCepts): InfoCepts is a fast-growing data and analytics solutions provider, recognized as a trusted partner for global enterprises. Headquartered in the US, InfoCepts has a strong delivery presence in India, focusing on modern data platforms, AI/ML, and cloud transformation. The company leverages deep expertise in Databricks, Snowflake, Azure, AWS, and GCP to help clients unlock the value of their data. With a collaborative culture and emphasis on innovation, InfoCepts offers a dynamic environment for technologists to solve real-world business challenges.

Role Overview: We are seeking an experienced Databricks Architect to join our team in Pune, Nagpur, Bangalore, or Chennai. As a Databricks Architect, you will be the technical authority for designing and delivering scalable, secure, and high-performance data solutions using the Databricks Lakehouse platform. You will lead architecture decisions, build robust data pipelines, and collaborate with stakeholders to translate business requirements into technical solutions. This role requires deep expertise in Apache Spark, cloud platforms (Azure/AWS/GCP), and modern data engineering practices.

Key Responsibilities:

• Architecture Design: Develop and maintain scalable, secure, and high-performance data architecture using Databricks, aligning with enterprise analytics goals.
• Data Pipeline Development: Lead the creation and optimization of ETL/ELT pipelines for both structured and unstructured data, ensuring reliability and efficiency.
• Cloud Integration: Seamlessly integrate Databricks with cloud platforms like Azure, AWS, or GCP, leveraging native services for storage, compute, and governance.
• Performance Optimization: Monitor and enhance the performance of data workflows and Apache Spark jobs, tuning for cost and speed.
• Stakeholder Collaboration: Work closely with data engineers, analysts, and business stakeholders to translate complex requirements into technical solutions.
• Mentorship & Guidance: Mentor junior team members, promote best practices, and drive a culture of continuous learning.
• Data Governance: Ensure data security, versioning using Delta Lake, and compliance with governance frameworks.
• Innovation: Stay ahead of technology trends, proactively adopting new tools such as MLflow, Airflow, and Kafka to enhance the platform.

Tech Stack:

• Core Platform: Databricks (Lakehouse Architecture), Apache Spark
• Languages: Python, Scala, SQL
• Data Storage & Management: Delta Lake, Data Versioning
• Cloud Platforms: Azure (primary), AWS, GCP (and their respective data services)
• DevOps/CI/CD: CI/CD pipelines, DevOps practices
• Orchestration: Apache Airflow (good to have)
• Streaming: Kafka or similar real-time data processing tools (good to have)
• Machine Learning: MLflow, ML model deployment (good to have)
• BI Tools: Power BI, Tableau (good to have)
• Certifications: Databricks or cloud certifications (good to have)

Minimum 10+ years in data engineering/architecture roles, with at least 3 years on Databricks. Bachelor’s or master’s degree in Computer Science, IT, or related field.

Full-time employment with opportunities to work on cutting-edge data platforms, mentor junior talent, and collaborate with cross-functional teams in a fast-paced environment.