INSPYR Solutions
Data Engineer
INSPYR Solutions, Miami, Florida, us, 33222
Title:
Data EngineerLocation:
Miami or Tampa, FLSchedule : on-site in Miami or TampaDuration:
Contract to HireWork Authorization:
US Citizen or Green Card holderData Engineering & ETL : Utilize 10+ years of experience in ETL tools, with at least 5 years dedicated to Azure Data Factory (ADF), to design, code, implement, and manage multiple parallel data pipelines. Additional knowledge about Fabric Pipelines and Data Flows Gen 2 usage is desirable.Data Manipulation : Develop and optimize data fetching, data wrangling, manipulation, and transformation processes, ensuring high-quality data delivery.SQL Proficiency : Leverage a strong SQL background, particularly in T-SQL, to work with Azure SQL Databases, performing complex queries, data transformations and able to perform basic database administration and access control configurations.Python & PySpark : Use Python, especially PySpark notebooks to convert and manage data. Understanding of Spark Streaming and Spark configuration is highly desirable. Hand on usage of Panda Libraries, Literal strings constants, user defined functions using Python. Experience in developing Python-based solutions is essential.Data Warehousing & Modeling : Apply a deep understanding of data warehousing concepts, including data modeling techniques like star and snowflake schemas, SCD Type 2, Change Data Feeds, Change Data Capture. Also demonstrates hands-on experience with Data Lake Gen 2, Delta Lake, Delta Parquet files, JSON files, big data storage layers, optimize and maintain big data storage using Partitioning, V-Order, Optimize, Vacuum and other techniques.Reporting Knowledge : Basic understanding of reporting tools, Semantic Layers, Tabular Models, Direct Lake Mode vs Incremental refreshes with Power BI experience considered a plus.API Integration & Event-Driven Architecture : Develop and manage REST API calls, webhooks, and event-driven architectures using Python. Experience with Azure Service Bus and Azure Event Hubs/ Event Grids/ Data Explorer is desirable.Schema Drift & ETL Architecture : Expertise in managing schema drift within ETL processes, ensuring robust and adaptable data integration solutions.Qualifications:
10+ years of experience in ETL tools, with a minimum of 5 years in Azure Data Factory (ADF),Dataflows Gen 2.Strong SQL skills with hand on experience with complex window functions, dynamic Sql, partitions, CDC,CDF particularly in T-SQL, and experience with Azure SQL Database, database administration and setup.Proficiency in Python, specifically with PySpark notebooks, for data processing tasks, configuration of Spark Pools, understanding of spark compute and consumption.Solid understanding of data warehousing concepts, data modeling (star and snowflake schemas), and SCD Type 2, CDC, CDF. Also implementation experience with Delta Lakes and Delta Parquet, JSON, data storage, optimization techniques for Big Data.Familiarity with REST API, webhooks, and event-driven architectures, Data Explorer, Functions Apps, Service Bus .Knowledge of reporting tools, query language, semantic models with a preference for Power BI experience.Ability to manage schema drift in semi structured files and adapt ETL architectures accordingly.Understanding of constructs like Lakehouse, EventHouse, KustoDB, KQL, Data Activator, Reflex is plus.
Our benefits package includes:
Comprehensive medical benefitsCompetitive pay401(k) retirement plan...and much more!
About INSPYR Solutions
Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.
INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.
Data EngineerLocation:
Miami or Tampa, FLSchedule : on-site in Miami or TampaDuration:
Contract to HireWork Authorization:
US Citizen or Green Card holderData Engineering & ETL : Utilize 10+ years of experience in ETL tools, with at least 5 years dedicated to Azure Data Factory (ADF), to design, code, implement, and manage multiple parallel data pipelines. Additional knowledge about Fabric Pipelines and Data Flows Gen 2 usage is desirable.Data Manipulation : Develop and optimize data fetching, data wrangling, manipulation, and transformation processes, ensuring high-quality data delivery.SQL Proficiency : Leverage a strong SQL background, particularly in T-SQL, to work with Azure SQL Databases, performing complex queries, data transformations and able to perform basic database administration and access control configurations.Python & PySpark : Use Python, especially PySpark notebooks to convert and manage data. Understanding of Spark Streaming and Spark configuration is highly desirable. Hand on usage of Panda Libraries, Literal strings constants, user defined functions using Python. Experience in developing Python-based solutions is essential.Data Warehousing & Modeling : Apply a deep understanding of data warehousing concepts, including data modeling techniques like star and snowflake schemas, SCD Type 2, Change Data Feeds, Change Data Capture. Also demonstrates hands-on experience with Data Lake Gen 2, Delta Lake, Delta Parquet files, JSON files, big data storage layers, optimize and maintain big data storage using Partitioning, V-Order, Optimize, Vacuum and other techniques.Reporting Knowledge : Basic understanding of reporting tools, Semantic Layers, Tabular Models, Direct Lake Mode vs Incremental refreshes with Power BI experience considered a plus.API Integration & Event-Driven Architecture : Develop and manage REST API calls, webhooks, and event-driven architectures using Python. Experience with Azure Service Bus and Azure Event Hubs/ Event Grids/ Data Explorer is desirable.Schema Drift & ETL Architecture : Expertise in managing schema drift within ETL processes, ensuring robust and adaptable data integration solutions.Qualifications:
10+ years of experience in ETL tools, with a minimum of 5 years in Azure Data Factory (ADF),Dataflows Gen 2.Strong SQL skills with hand on experience with complex window functions, dynamic Sql, partitions, CDC,CDF particularly in T-SQL, and experience with Azure SQL Database, database administration and setup.Proficiency in Python, specifically with PySpark notebooks, for data processing tasks, configuration of Spark Pools, understanding of spark compute and consumption.Solid understanding of data warehousing concepts, data modeling (star and snowflake schemas), and SCD Type 2, CDC, CDF. Also implementation experience with Delta Lakes and Delta Parquet, JSON, data storage, optimization techniques for Big Data.Familiarity with REST API, webhooks, and event-driven architectures, Data Explorer, Functions Apps, Service Bus .Knowledge of reporting tools, query language, semantic models with a preference for Power BI experience.Ability to manage schema drift in semi structured files and adapt ETL architectures accordingly.Understanding of constructs like Lakehouse, EventHouse, KustoDB, KQL, Data Activator, Reflex is plus.
Our benefits package includes:
Comprehensive medical benefitsCompetitive pay401(k) retirement plan...and much more!
About INSPYR Solutions
Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.
INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.