MarketWise
AI/ML Data Engineer
MarketWise, Baltimore, Maryland, United States, 21276
Key Responsibilities
Develop and maintain data pipelines and ETL processes using Python, Pandas, and Apache Spark.Design and implement scalable data processing solutions on cloud platforms such as AWS, Azure, or Google Cloud Platform.Collaborate with data scientists and analysts to understand data requirements and ensure smooth data delivery and availability.Perform data cleaning, transformation, and normalization tasks to ensure data integrity and quality.Optimize data workflows and infrastructure for performance and efficiency.Implement data governance and security best practices to protect sensitive information.Proactively identify opportunities for data quality improvements and implement solutions.
Required Skills And Qualifications
Proficiency in Python programming language with a strong emphasis on data manipulation using Pandas.Experience with Apache Spark for large-scale data processing and distributed computing.Hands-on experience with cloud computing platforms (AWS, Azure, GCP) and services such as S3, EC2, EMR, Databricks, or equivalent.Solid understanding of data cleaning, formatting, and transformation techniques.Familiarity with SQL and NoSQL databases for data storage and retrieval.Excellent problem-solving skills and the ability to work independently or as part of a team.Strong communication skills and the ability to effectively collaborate with cross-functional teams.
Preferred Qualifications
Background or experience in finance or financial data analysis.Knowledge of machine learning frameworks and libraries (e.g., TensorFlow, scikit-learn) for data modeling and analysis.
#J-18808-Ljbffr
Develop and maintain data pipelines and ETL processes using Python, Pandas, and Apache Spark.Design and implement scalable data processing solutions on cloud platforms such as AWS, Azure, or Google Cloud Platform.Collaborate with data scientists and analysts to understand data requirements and ensure smooth data delivery and availability.Perform data cleaning, transformation, and normalization tasks to ensure data integrity and quality.Optimize data workflows and infrastructure for performance and efficiency.Implement data governance and security best practices to protect sensitive information.Proactively identify opportunities for data quality improvements and implement solutions.
Required Skills And Qualifications
Proficiency in Python programming language with a strong emphasis on data manipulation using Pandas.Experience with Apache Spark for large-scale data processing and distributed computing.Hands-on experience with cloud computing platforms (AWS, Azure, GCP) and services such as S3, EC2, EMR, Databricks, or equivalent.Solid understanding of data cleaning, formatting, and transformation techniques.Familiarity with SQL and NoSQL databases for data storage and retrieval.Excellent problem-solving skills and the ability to work independently or as part of a team.Strong communication skills and the ability to effectively collaborate with cross-functional teams.
Preferred Qualifications
Background or experience in finance or financial data analysis.Knowledge of machine learning frameworks and libraries (e.g., TensorFlow, scikit-learn) for data modeling and analysis.
#J-18808-Ljbffr