Job Title

WHAT THIS ROLE WILL DOThis is a hands-on Data role with strong Python/PySpark coding skills Contribute to the enhancements and continuous improvement of performance and reliability of the existing Data Platform Services to meet the requirements from Data Engineering/Product teamDesign and build self-servicing onboarding and automation capabilities in Core Data Platform services in a scalable, reliable and secure way.Design and build platform capabilities to support onboarding and operation effectiveness Design and develop scalable data platform and integration solutions for various data sources leveraging Databricks Unified PlatformEnsure data quality and integrity across all data systems.Develop and maintain documentation for data systems and processes.Monitor and troubleshoot data platform issues.Participate in on-call rotations/Pagerduty for data platform support.TECHNICAL SKILLS/COMPETENCIESGood understanding of Data Lakes, Data Warehouses is MUST to haveSoftware development, coding expertise in the Data Engineering field using any of the Python/Pandas/PySpark is must to haveHands on experience using version control systems such as Git and CI/CD workflows and practiceDesign and Develop data ingestion services that are highly performant, reliable and scalableDecent expertise using ANSI SQL and Spark SQL is key to have Workflow automation, orchestration using Airflow or equivalent tools stackHands on working knowledge on at least in one of these: Databricks, Hadoop and related stacksWorking experiencein at least one of cloud services from any of the Amazon AWS, Google GCP or Microsoft Azure, preferably AWSstreaming (Kafka) and batch-based data sourcesdiverse data sources and data formats (xml, json, yaml, parquet, avro, delta) and respective use casesAny visualization experience is an advantage so you can bring clarity to prod incidents by using them is a nice to haveAn excellent understanding of the nuances that add complexities around time zones, geo, various data formats, data types across different storage systems is nice to haveAgile development methodologies using the Atlassian suite: Jira, ConfluenceCollaborate with cross-functional teams to deliver data solutions and provide technical support and guidance to team members.Stay up to date with the latest data engineering technologies and trends.