Data Engineer

Data EngineerThis role requires a blend of technical expertise in data engineering, full-stack development skills, and a strong understanding of data architectures. The ideal candidate will be able to bridge the gap between traditional SQL environments and modern big data platforms, while also contributing to the full stack of data-driven applications.Data Engineer Technical Skills Required:Expert proficiency in Python, C#, and T-SQLStrong experience with Databricks and Apache Spark (especially Spark SQL)Proficient in SQL Server and ASP.NET CoreExperience with Apache KafkaSkilled in query optimization and SQL performance tuningFamiliarity with data warehousing concepts and dimensional modelingKnowledge of containerization technologies (e.g., Docker, Kubernetes)Experience with cloud platforms (preferably Azure)Proficiency in Git and CI/CD pipelinesFamiliarity with data governance and security best practicesUnderstanding of RESTful API design principlesExperience with data visualization tools (e.g., Power BI, Tableau)Data Engineer Responsibilities:Design, build, and maintain scalable data jobs and workflows in Databricks using Spark SQL and PythonMigrate and optimize SQL stored procedures from T-SQL to DatabricksDevelop and maintain RESTful backend APIs using C# and ASP.NET CoreImplement advanced data transformations, aggregations, and analytical workloads using Spark SQLCollaborate with data scientists and analysts to deliver optimized data solutionsImplement and maintain data governance practices, ensuring data quality, integrity, and securityOptimize Databricks jobs and SQL queries for performance and cost-effectivenessAutomate data workflows using Python scripting and orchestration toolsDesign and implement data pipelines for real-time and batch processingContribute to the design and maintenance of data warehouses and data lakesParticipate in code reviews and mentor junior team membersStay current with emerging data engineering technologies and best practicesTroubleshoot and resolve complex data-related issuesDevelop documentation for data processes, APIs, and architecturesCollaborate with cross-functional teams to integrate data solutions into broader applications and services