Data Engineer (Remote) Job at ResolveSoft Inc in Washington

Responsibilities:

Data Engineering & ETL Development:

• Design and implement scalable and efficient ETL pipelines using Databricks, Apache Spark, and other cloud-based technologies.
• Ensure seamless data ingestion, transformation, and storage to meet analytical and operational requirements.

Data Architecture:

• Develop and optimize data architecture for high-performance analytics and reporting.
• Build and maintain Lakehouse architectures using Databricks and cloud platforms (AWS, Azure, or GCP).

Collaboration & Integration:

• Work collaboratively with Data Scientists, Analysts, and Software Engineers to build data products and solutions.
• Integrate data from diverse sources, including structured, semi-structured, and unstructured data.

Performance Tuning:

• Monitor, troubleshoot, and optimize data workflows and Databricks jobs for performance and cost-efficiency.

Data Governance & Security:

• Implement best practices in data governance, security, and compliance.
• Maintain data integrity and reliability through robust testing and monitoring frameworks.

Automation & CI/CD:

• Automate data pipelines and workflows using CI/CD practices and tools like Git, Azure DevOps, or Jenkins.

Qualifications:

• Must have 3-10 years of hands-on experience in data engineering and data pipeline building.
• Experience with Databricks and Apache Spark.
• Public Trust Clearance
• Must be a U.S. citizen.

Technical Skills:

• Proficiency in programming languages like Python, Scala, or Java.
• Strong understanding of SQL and relational databases (e.g., PostgreSQL, MySQL).
• Experience with cloud platforms such as AWS (Glue, S3), Azure (Data Factory, ADLS), or Google Cloud (BigQuery, Dataflow).
• Knowledge of data modeling, data warehousing, and ELT/ETL best practices.
• Experience with metrics formulation and dashboarding.
• Experience with Qlik is a plus.

Tools & Technologies:

• Experience with orchestration tools (e.g., Airflow, Apache NiFi, Prefect).
• Familiarity with BI tools like Tableau, Power BI, or Looker.

Soft Skills:

• Excellent problem-solving abilities and the capacity to thrive in a fast-paced, agile work environment.
• Exhibit strong communication and collaboration skills that connect technical and non-technical stakeholders, fostering effective engagement and achieving successful outcomes.