Highbrow LLC
AWS Data Bricks Spark/Scala Engineer
Highbrow LLC, Saint Louis, Missouri, United States, 63146
Job Title:
AWS Data Bricks Spark/Scala Engineer
Job ID:
2023-11933
Job Location:
Morris Plains, NJ (remote to start); Remote, Memphis, TN (preferred), Morris Plains, NJ, Austin, TX (preferred), St. Louis, MO
Employment Type:
W2
Duration:
Long Term
Work Eligibility:
All Work Authorizations are Permitted
Key Technology:
DataBricks, AWS, Spark
Job Responsibilities:
Drive the mentality of building well-architected applications for AWS Cloud.
Promote quality ownership across the entire team.
Identify code defects and collaborate with developers to address quality issues in product code.
Find bottlenecks and thresholds in existing code using automation tools.
Articulate clear business objectives aligned with technical specifications and work in an iterative agile pattern daily.
Take ownership of tasks and engage with all team levels, raising challenges when necessary.
Skills and Experience Required:
6-8 years of strong experience in Spark, Python, Shell Scripting, PostgreSQL, Hadoop, AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and CloudWatch), Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration).
In-depth knowledge of Hadoop architecture and its components, with the ability to troubleshoot Hadoop Yarn jobs. Experience in implementing Hadoop job orchestration using Shell scripting and Airflow.
In-depth understanding of Spark Architecture, including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, and Spark MLib.
Strong experience in writing complex and effective SQLs and Stored Procedures, with query optimization skills to improve ETL performance.
Hands-on cloud experience with the AWS suite.
Demonstrated work experience in Hadoop, Spark, and Hive.
Strong development experience on Databricks components.
Significant hands-on experience with Spark.
Experience with Hadoop as a data warehouse/Data Lake implementation.
Moderate to senior Terraform development experience.
Health care domain knowledge is a plus.
#J-18808-Ljbffr
AWS Data Bricks Spark/Scala Engineer
Job ID:
2023-11933
Job Location:
Morris Plains, NJ (remote to start); Remote, Memphis, TN (preferred), Morris Plains, NJ, Austin, TX (preferred), St. Louis, MO
Employment Type:
W2
Duration:
Long Term
Work Eligibility:
All Work Authorizations are Permitted
Key Technology:
DataBricks, AWS, Spark
Job Responsibilities:
Drive the mentality of building well-architected applications for AWS Cloud.
Promote quality ownership across the entire team.
Identify code defects and collaborate with developers to address quality issues in product code.
Find bottlenecks and thresholds in existing code using automation tools.
Articulate clear business objectives aligned with technical specifications and work in an iterative agile pattern daily.
Take ownership of tasks and engage with all team levels, raising challenges when necessary.
Skills and Experience Required:
6-8 years of strong experience in Spark, Python, Shell Scripting, PostgreSQL, Hadoop, AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and CloudWatch), Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration).
In-depth knowledge of Hadoop architecture and its components, with the ability to troubleshoot Hadoop Yarn jobs. Experience in implementing Hadoop job orchestration using Shell scripting and Airflow.
In-depth understanding of Spark Architecture, including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, and Spark MLib.
Strong experience in writing complex and effective SQLs and Stored Procedures, with query optimization skills to improve ETL performance.
Hands-on cloud experience with the AWS suite.
Demonstrated work experience in Hadoop, Spark, and Hive.
Strong development experience on Databricks components.
Significant hands-on experience with Spark.
Experience with Hadoop as a data warehouse/Data Lake implementation.
Moderate to senior Terraform development experience.
Health care domain knowledge is a plus.
#J-18808-Ljbffr