V-Soft Consulting Group, Inc.
Data Engineer
V-Soft Consulting Group, Inc., Dallas, TX, United States
Title: Data Engineer
Required Location: Dallas, TX (and in-office 5 days a week. Candidates must be local and available to interview in-person.
- TOP SKILLS:Although this role will be sitting on-site, they must also be able to use a personal workstation, or options are available for rent. They are a Microsoft shop.
- 5-8+ years of experience in data engineering or a related role.
- Seeking an individual with the ability to learn fast, eager to be in a collaborative team environment and work with an offshore team.
- Design, build, and manage ETL data pipelines using AWS services services, specifically Glue, Athena, and Lambda is a plus.
- Hands-on experience with Apache Hudi for real-time data management.
- Strong programming skills in Python and PySpark.
- Familiarity with SQL and NoSQL databases.
- Knowledge of data governance and data security best practices.
- Experience with workflow scheduling tools like AWS Step Functions or Control M.
- Collaborate with cross-functional teams to meet business objectives.
- Monitor performance and advise on any necessary infrastructure changes.
- Experience with Hadoop ecosystem is a must.
- Experience with RedShift is a must.
- Develop technical documentation including data dictionaries, metadata