Job Title

Director at Kaliba - Securing APAC''s top talent for Global Software CompaniesMid to Senior Java/Scala Developer Open Lakehouse & Apache SparkAre you a skilled Java/Scala Developer with a passion for distributed systems and large-scale data processing? Were working with an innovative company at the forefront of open-source data technologies, looking for a Mid to Senior Java/Scala Developer to join their team. This role focuses on building and optimizing distributed applications using Apache Spark and modern Lakehouse architectures.While mid to senior-level experience is preferred, we welcome applications from high-achieving Masters graduates in relevant fields.Key Responsibilities:Software Development: Design, develop, and optimize distributed applications using Java and Scala for large-scale data processing.Apache Spark Expertise: Utilize deep knowledge of Spark internals to build efficient, scalable data pipelines.Table Formats: Work with open-source table formats like Apache Iceberg, Delta Lake, Apache Hudi, and Apache Xtable.Lakehouse Architecture: Implement open Lakehouse solutions, including cataloging systems like Unity Catalog and Polaris Catalog, and manage ML workflows with MLflow.Computer Science Fundamentals: Apply expertise in data structures, caching, networking, and databases to build high-performance systems.Collaboration: Work closely with data engineers, machine learning engineers, and business stakeholders to develop cutting-edge solutions.Key Requirements:Experience: 3+ years in Java or Scala development (or a Masters degree in Computer Science or related field).Strong CS Knowledge: Solid understanding of data structures, caching strategies, networking, and database management.Apache Spark: Deep understanding of Spark execution, query optimization, and distributed data processing.Open Source Table Formats: Hands-on experience with Apache Iceberg, Delta Lake, Apache Hudi, or Apache Xtable.Open Lakehouse & MLflow: Familiarity with Unity Catalog, Polaris Catalog, and MLflow for data and ML integration.Problem-Solving: Ability to troubleshoot and optimize large-scale distributed systems.Preferred Qualifications:Experience with cloud platforms (AWS, GCP, Azure).Familiarity with CI/CD pipelines, Git, and GitHub.Understanding of machine learning workflows in data pipelines.If youre passionate about large-scale data systems and open-source technologies, this role offers a chance to work with an innovative and forward-thinking team.Interested? Apply now or reach out to learn more!Seniority levelMid-Senior levelEmployment typeFull-timeJob functionInformation TechnologyIndustriesSoftware Development, Financial Services, and IT Services and IT Consulting #J-18808-Ljbffr