Data Engineering Manager - Experience required- 10years and above Must Haves - 1. Database Architecture 2. Data modeling 3. Big data development 4. ETL development 5. Data warehousing 6. Data provisioning 1. Python coding knowledge 2. ETL processes 3. SQL 4. python Building near real time , data pipeline using kafka spring boot and load data into DB. Incorporate all SecDevOps related things to be taken care as part of development i.e deploy into Kubernates, building monitoring , alerting , distributed tracing , self healing etc. Writing stored procedure and queries to do data profiling and reconciliation of data loaded as part of Near Real Time. building data model with security to expose data to different consumers Maintain, improve, clean, and manipulate data in the business operational and analytics databases Design and deploy data platforms across multiple domains ensuring operability Transform data for meaningful analyses Improve data efficiency, reliability and quality Create data enrichment Build high performance Ensure data integrity Create and manage data stores at scale Ensure data governance - security, quality, access and compliance Manage performance of high-performing engineers and promote technical design discussions , Jointly explore technical solutions to problems & Perform code reviews - Undergraduate degree or equivalent experience. Qualifications - External BS/BA or equivalent experience 12+ years of overall experience designing, coding and supporting distributed, data intensive systems at scale 10+ years of experience with relational database delivery - Oracle, SQL Server & mysql 8+ years of experience working within the Software Development Life Cycle (SDLC) 3+ years of experience in Agile Delivery 7+ Hands-on experience with ETL tools ( Databricks , Snowflake & Airflow , Kafka ) 3+ years of Experience with programming languages (Python, Java, Go, scala, etc) 2+ years of Experience in delivering Data Platforms involving Snowflake cloud DW 3+ years of Hands-on experience of big data and streaming frameworks - Kafka , Spark , Databricks 2+ years of. Experience and working exposure in cloud environment, preferably Azure
Job Title
Data Engineer