International Executive Service Corps
Data Engineer 4
International Executive Service Corps, Charlotte, North Carolina, United States, 28245
Data Engineer 4
Join our dynamic team at our energy partner as a Data Engineer 4, where you will play a pivotal role in harnessing the power of data to optimize operations, enhance decision-making processes, and drive innovation in the rapidly evolving energy sector. As a seasoned professional, you will leverage your expertise to design, implement, and maintain robust data pipelines, ensuring the seamless flow of information critical to powering the future of sustainable energy solutions.
Core Responsibilities
Provides technical direction, guides the team on key technical aspects and is responsible for product tech delivery.
Leads the design, build, test, and deployment of components.
Collaborates with lead developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead) where applicable.
Understands requirements/use cases to outline technical scope and lead delivery of technical solutions.
Confirms required developers and skillsets specific to product.
Provides leadership, direction, peer review, and accountability to developers on the product (key responsibility).
Works closely with the Product Owner to align on delivery goals and timing.
Assists Product Owner with prioritizing and managing team backlog.
Collaborates with Data and Solution architects on key technical decisions.
Architects and designs to deliver the requirements and functionality.
Core Experience and Abilities
Power Systems engineering experience.
DER Dispatch experience.
Automated fault analysis systems experience.
Ability to perform hands-on development and peer review for certain components/tech stack on the product.
Standing up of development instances and migration path (with required security, access/roles).
Develops components and related processes (e.g., data pipelines and associated ETL processes, workflows).
Leads implementation of integrated data quality framework.
Ensures optimal framework design and load testing scope to optimize performance (specifically for Big Data).
Supports data scientists with test and validation of models.
Performs impact analysis and identifies risk to design changes.
Ability to build new data pipelines, identify existing data gaps, and provide automated solutions to deliver analytical capabilities and enriched data to applications.
Ensures Test Driven Development.
Experience leading teams to deliver complex products.
Strong technical and communication skills.
Strong skills with business stakeholder interactions.
Strong solutioning and architecture skills.
Experience building real-time data ingestion streams (event-driven).
Ensures data security and permissions solutions, including data encryption, user access controls, and logging.
Core Technical Skills
Experience with native AWS technologies for data and analytics such as Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch, etc.
Tools and languages such as Django, Python, Java, Scala, Pandas.
Infrastructure as Code technology such as Terraform.
AWS services such as S3, EMR, Glue, Lambda, Athena, Kinesis, EC2, SNS, SQS, CloudWatch.
Experience with databases such as Redshift, Document DB, DynamoDB, and MongoDB.
Experience transitioning on-premise big data platforms into cloud-based platforms such as AWS.
Additional Technical Skills (nice to have, but not required for the role)
Hadoop platform (Hive; HBase; Druid).
Spark.
PySpark.
SQL.
Workflow Automation.
DevOps pipeline (CI/CD); Bitbucket; Concourse.
API frameworks.
#J-18808-Ljbffr
Join our dynamic team at our energy partner as a Data Engineer 4, where you will play a pivotal role in harnessing the power of data to optimize operations, enhance decision-making processes, and drive innovation in the rapidly evolving energy sector. As a seasoned professional, you will leverage your expertise to design, implement, and maintain robust data pipelines, ensuring the seamless flow of information critical to powering the future of sustainable energy solutions.
Core Responsibilities
Provides technical direction, guides the team on key technical aspects and is responsible for product tech delivery.
Leads the design, build, test, and deployment of components.
Collaborates with lead developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead) where applicable.
Understands requirements/use cases to outline technical scope and lead delivery of technical solutions.
Confirms required developers and skillsets specific to product.
Provides leadership, direction, peer review, and accountability to developers on the product (key responsibility).
Works closely with the Product Owner to align on delivery goals and timing.
Assists Product Owner with prioritizing and managing team backlog.
Collaborates with Data and Solution architects on key technical decisions.
Architects and designs to deliver the requirements and functionality.
Core Experience and Abilities
Power Systems engineering experience.
DER Dispatch experience.
Automated fault analysis systems experience.
Ability to perform hands-on development and peer review for certain components/tech stack on the product.
Standing up of development instances and migration path (with required security, access/roles).
Develops components and related processes (e.g., data pipelines and associated ETL processes, workflows).
Leads implementation of integrated data quality framework.
Ensures optimal framework design and load testing scope to optimize performance (specifically for Big Data).
Supports data scientists with test and validation of models.
Performs impact analysis and identifies risk to design changes.
Ability to build new data pipelines, identify existing data gaps, and provide automated solutions to deliver analytical capabilities and enriched data to applications.
Ensures Test Driven Development.
Experience leading teams to deliver complex products.
Strong technical and communication skills.
Strong skills with business stakeholder interactions.
Strong solutioning and architecture skills.
Experience building real-time data ingestion streams (event-driven).
Ensures data security and permissions solutions, including data encryption, user access controls, and logging.
Core Technical Skills
Experience with native AWS technologies for data and analytics such as Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch, etc.
Tools and languages such as Django, Python, Java, Scala, Pandas.
Infrastructure as Code technology such as Terraform.
AWS services such as S3, EMR, Glue, Lambda, Athena, Kinesis, EC2, SNS, SQS, CloudWatch.
Experience with databases such as Redshift, Document DB, DynamoDB, and MongoDB.
Experience transitioning on-premise big data platforms into cloud-based platforms such as AWS.
Additional Technical Skills (nice to have, but not required for the role)
Hadoop platform (Hive; HBase; Druid).
Spark.
PySpark.
SQL.
Workflow Automation.
DevOps pipeline (CI/CD); Bitbucket; Concourse.
API frameworks.
#J-18808-Ljbffr