Role: Data EngineerExperience: 5-10Description:We are seeking a skilled Databricks Data Engineer with expertise in Scala to design, implement, and maintain scalable data solutions on the Databricks platform. The ideal candidate will have a strong background in data engineering, distributed computing, and cloud technologies, with a focus on leveraging Databricks and Apache Spark for data processing and analytics. The project in which the person will be assigned is a data engineering effort to support one of clients with timeseries data migration.Responsibilities:Design and Implementation: Develop and maintain robust data pipelines using Databricks Scala, ensuring efficient data processing and transformation.Data Modeling: Create and optimize data models to support analytical workloads and reporting needs.Collaboration: Work closely with cross-functional teams, including analysts, and business stakeholders, to deliver data-driven solutions that meet business objectives.Performance Optimization: Monitor and enhance the performance of existing data pipelines and applications, implementing best practices for scalability and efficiency.ETL Development: Design and implement ETL processes to extract, transform, and load data from various sources into Databricks.Automation: Utilize DevOps practices to automate deployment processes and monitor data pipelines for reliability.Documentation: Maintain clear documentation of data architecture, processes, and workflows to ensure knowledge sharing within the team.Skills: Programming Languages: Proficiency in Scala is required; experience with Python or SQL is a plus.Databricks Experience: Hands-on experience with the Databricks platform and Apache Spark for big data processing.Cloud Platforms: Strong knowledge of Azure in relation to data services.Distributed Computing: Understanding of distributed computing principles and big data technologies.Data Warehousing: Familiarity with data warehousing concepts and tools.Timeseries DB: Knowledge of timeseries DB like influx dbColumnar Data store: Knowledge of columnar datastore like Apache Druid or Imply Polaris
Job Title
Concentrix | Data Engineer