ETL Engineer

Serve as a core member of a project team, providing design, coding, configuration and operational support of key data processes within the OptiMine Insight and Intent environments. Ensure the quality and increasing scale of automated data onboarding operations for OptiMine clients and client success team members to support complex and sophisticated data operations on behalf of our customers. Design, develop, enhance, migrate, test, and maintain new data warehousing application solutions using ETL ( Extract, Transform and Load ), a data pipeline used to collect data from various client sources, transform the data according to defined business rules, and load the data into the destination data warehouse. Specific duties include: Collaborate with Technical Project Management, Account Management, and Software Engineers to analyze client business requirements: o Analyze and interpret complex data from target (client) systems. o Translate and validate client business requirements. o Evaluate client data load/change requests, and recommend and develop solutions. Analyze and improve OptiMine s ETL application solutions: o Analyze existing ETL processes (both manual and automated) for patterns, scalability and efficiency improvements, and design new automated ETL routines for continuous improvement. o Monitor data production jobs and maintain and enhance legacy ETL processes on a regular basis. o Improve slow running jobs by redesigning more efficient ETL processes to meet business needs. o Working in collaboration with OptiMine engineering and client success teams, design, model, develop, and maintain existing and new SQL database objects required for all business solutions, ensuring all technical work product is thoroughly tested for reliability, scale and repeatability. Develop Data Warehouse Model: o Define dimensions and standards (organization, structure, nomenclature). o Design data warehouse structures (architecture, tools, databases). o Participate with software development teams to construct data warehouse. Map data path for ETL, create ETL data functions, and document process flow: o Prepare technical reports and system specification documents for meta data integration. o Diagram data flow between source systems and data warehouse, and create ETL schedule. o Write scripts and programs to execute ETL using software programming languages and technologies. Execute ETL processes: o Extraction: Run software scripts and programs for data extraction from client source system. o Transformation: Validate and perform quality analysis on inbound data sets, and monitor data transformation operations, including filtering, sorting, aggregating, joining, cleaning, and deduplication. o Load: Monitor and analyze migration of data from transformation process into warehouse (databases), and perform tests during load to ensure control of data flow and data standards. Test and troubleshoot ETL processes throughout project lifecycle: o Develop and document test plans, files and procedures for systems and ETL processes. o Perform root cause analysis on all processes, and develop and implement technical resolution actions. Perform SQL queries and Data Exports as needed: o Conduct system analysis and routine tests on data warehouses. o Prepare technical documentation. Position involves the use of the following tools and technologies: MS SQL Server, Python, Unix, Java/JavaScript, and JSON. Position is based in Minneapolis, Minnesota. Telecommuting allowed with at least one day per week in office required. Minimum Requirements: Master s degree in Computer Science, Information Assurance, or related technical field; plus at least 1 year of application development experience, including ETL data warehousing experience with development, migration, and testing duties using MS SQL Server. Alternatively, will accept Bachelor s degree in Computer Science, Information Assurance, or related technical field; plus at least 3 years of application development exper ence, including at least 1 year ETL data warehousing experience with development, migration, and testing duties using MS SQL Server. Email resume to careers@optimine.com OR mail resume to Attn: OptiMine Careers, OptiMine Software, Inc., 1400 Van Buren St. NE, Suite 200, Minneapolis, MN 55413. Reference job title.