Mapfre
Data Engineer
Mapfre, Webster, Massachusetts, us, 01570
Job Summary
Augment and maintain the existing repositories and data structures within AWS (used to process and store large amounts of data from unrelated sources)Experience with several formats and means for data ingestion. Including, data types (structured, semi-structured, and unstructured), and sources (on premise, and in the cloud) using the most appropriate techniques in each caseContinue to expand and enhance the model, utilizing best practices, in regards to the organization of data and the various relationshipsOptimize existing and future models for fast and scalable queries (while maintaining performance and related price thresholds)Work with the team to define, construct, and maintain self-service dashboards for the Business and Advanced Analytics teams within PowerBIImplement scalable and flexible, high performance data pipelines for AWS to support analyticsDevelop and maintain data maps and their relationshipsGenerate associated technical documentation including follow-up reportsWork with Data Governance to implement quality rules and data governance measures (data dictionary, metadata, traceability, ...)Propose improvements and actions based on provided resultsCommunicate results effectively with required teamsKnowledge, Skills and Abilities:
Bachelor's Degree with 6+ years of experience.Advanced knowledge and experience using Python, Airflow, Spark, AWS, and SnowflakeDatabase architectures: SQL, NoSQL, graph databasesCI/CD and Orchestration: Jira, Jenkins, Bit Bucket, Terraform, and AirflowPast experience with data modeling tools, ETL tools (e.g. Informatica Power Center)Computer languages, data query and transformation tools: AWS Athena, Jupyter notebooks, Spark, Pyspark, Python, and EMR StudioAlgorithm analysis (for working with our Data Scientists)Understanding of multidimensional modeling for quantitative and fact related data storageOS: Linux, and MS WindowsCode IDE: Microsoft Visual Code, Jupyter notebooksArtificial intelligence, machine learning, and deep learning are a plus
Augment and maintain the existing repositories and data structures within AWS (used to process and store large amounts of data from unrelated sources)Experience with several formats and means for data ingestion. Including, data types (structured, semi-structured, and unstructured), and sources (on premise, and in the cloud) using the most appropriate techniques in each caseContinue to expand and enhance the model, utilizing best practices, in regards to the organization of data and the various relationshipsOptimize existing and future models for fast and scalable queries (while maintaining performance and related price thresholds)Work with the team to define, construct, and maintain self-service dashboards for the Business and Advanced Analytics teams within PowerBIImplement scalable and flexible, high performance data pipelines for AWS to support analyticsDevelop and maintain data maps and their relationshipsGenerate associated technical documentation including follow-up reportsWork with Data Governance to implement quality rules and data governance measures (data dictionary, metadata, traceability, ...)Propose improvements and actions based on provided resultsCommunicate results effectively with required teamsKnowledge, Skills and Abilities:
Bachelor's Degree with 6+ years of experience.Advanced knowledge and experience using Python, Airflow, Spark, AWS, and SnowflakeDatabase architectures: SQL, NoSQL, graph databasesCI/CD and Orchestration: Jira, Jenkins, Bit Bucket, Terraform, and AirflowPast experience with data modeling tools, ETL tools (e.g. Informatica Power Center)Computer languages, data query and transformation tools: AWS Athena, Jupyter notebooks, Spark, Pyspark, Python, and EMR StudioAlgorithm analysis (for working with our Data Scientists)Understanding of multidimensional modeling for quantitative and fact related data storageOS: Linux, and MS WindowsCode IDE: Microsoft Visual Code, Jupyter notebooksArtificial intelligence, machine learning, and deep learning are a plus