Uipathtek
Big Data Engineer
Uipathtek, Charlotte, North Carolina, United States, 28245
Location:
UIPATHTEK LLC, 8307 University Executive Park Drive, Suite #242, Charlotte, NC 28262Job Description
Working as part of the Big Data Engineering team which is responsible for transforming data into useful information for the data science team and product team. Working with Linux systems and Hadoop databases to extract data from Hadoop database and ingested using Scoop. Staging the real-time data from gateway into AWS S3 or Azure Blob storage. Analyzing data using SQL Queries and transforming data into various stages such as preprocessed, standardized, and filtered. Implementing Spark using Scala and utilizing Data frames and Spark SQL API for faster processing of data. Responsible to use partitions in Spark session to improve the performance of the load time. Creating Pipelines in ADF using Linked Services/Datasets/Pipeline to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards. Responsible for importing data and developing Spark streaming pipeline in Java. Work under supervision. Travel and/or relocation to unanticipated client sites is required.Education Required
Master's degree in Computer Science/IT/IS/Engineering (Any) or Closely Related field.Experience Required
Please see the Job description.Contact Information:Phone: +1 980-248-9633Email: info@uipathtek.comHead Quarters: 8307 University Executive Park Dr, Suite #242, Charlotte, NC 28262
#J-18808-Ljbffr
UIPATHTEK LLC, 8307 University Executive Park Drive, Suite #242, Charlotte, NC 28262Job Description
Working as part of the Big Data Engineering team which is responsible for transforming data into useful information for the data science team and product team. Working with Linux systems and Hadoop databases to extract data from Hadoop database and ingested using Scoop. Staging the real-time data from gateway into AWS S3 or Azure Blob storage. Analyzing data using SQL Queries and transforming data into various stages such as preprocessed, standardized, and filtered. Implementing Spark using Scala and utilizing Data frames and Spark SQL API for faster processing of data. Responsible to use partitions in Spark session to improve the performance of the load time. Creating Pipelines in ADF using Linked Services/Datasets/Pipeline to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards. Responsible for importing data and developing Spark streaming pipeline in Java. Work under supervision. Travel and/or relocation to unanticipated client sites is required.Education Required
Master's degree in Computer Science/IT/IS/Engineering (Any) or Closely Related field.Experience Required
Please see the Job description.Contact Information:Phone: +1 980-248-9633Email: info@uipathtek.comHead Quarters: 8307 University Executive Park Dr, Suite #242, Charlotte, NC 28262
#J-18808-Ljbffr