Logo
BGSF

Sr. Splunk Infrastructure Engineer

BGSF, Owings Mills, Maryland, United States, 21117


Sr. Splunk Infrastructure Engineer

Role Description

The Sr. Splunk Infrastructure Engineer will be responsible for supporting Splunk Enterprise, including managing Windows and Linux servers' agents, automating infrastructure, configuration, and day-to-day operations through Ansible, and Performing troubleshooting, root cause analysis, and resolution of complex technical issues related to Splunk deployments.

Responsibilities

• Support onboarding and maintenance of logs to Splunk from Windows, Linux, and cloud-based sources.

• Support platform upgrades including coordinating testing of upgrades with users of the platform.

• Automating manual platform management processes through Ansible or other scripting tools/languages

• Troubleshooting incidents impacting the Splunk platform

• Evaluate the use and integration of third-party add-ons.

• Coordinating and collaborating with users of the platform

• Develop training and documentation materials.

Experience General

• Ability to troubleshoot and diagnose complex issues.

• Able to demonstrate experience supporting technical users and conducting requirements analysis.

• Can work independently with minimal guidance & oversight.

• Experience with IT Service Management and familiarity with Incident & Problem management

• Highly skilled in identifying performance bottlenecks, identifying anomalous system behavior, and resolving the root cause of service issues.

• Demonstrated ability to effectively work across teams and functions to influence design, operations, and deployment of universally available software.

• Knowledge of standard methodologies related to security, performance, and disaster recovery.

Required Technical Expertise

• 3 years' experience managing and configuring Splunk Enterprise and/or Splunk Cloud

• Experience with Splunk clustered deployment topology

• Experience with Linux and Windows agents for Splunk administration

• Experience in designing, developing, and deploying cloud-based solutions using AWS.

• Experience in onboarding new data, configuration, creating new dashboards, and extracting information through Splunk.

• Experience with writing or modifying custom Splunk add-ons.

• Demonstrated proficiency with scripting and automation (bash, python, other programming languages)

• Familiarity with Splunk rest APIs

• Strong scripting skills (e.g., Python, Bash) for automation and custom development.

• In-depth knowledge of log management, data onboarding, and SIEM principles.

Preferred Technical Experience

• Splunk Certification (Admin or Architect)

• Experience with Ansible tower automation

• Experience using Gitlab.

• Experience with large platform migration efforts

• Experience with AWS OpenSearch

• Experience with Crib

• Expertise in languages such as Java, and Python. Implementation knowledge in data processing pipelines using programming languages like Java and Python to extract, transform, and load (ETL) data.

• Create and maintain data models, ensuring efficient storage, retrieval, and analysis of large datasets.

• Troubleshoot and resolve issues related to data processing, storage, and retrieval.

• 3-5 years' Experience in designing, developing, and deploying data lakes using AWS native services (S3, Glue (Crawlers, ETL, Catalog), IAM, Terraform, Athena)

• Experience in the development of systems for data extraction, ingestion, and processing of large volumes of data

• Experience with data pipeline orchestration platforms

• Experience in Ansible/Terraform/Cloud Formation scripts and Infrastructure as Code scripting is required.

• Implement version control and CI/CD practices for data engineering workflows to ensure reliable and efficient deployments.

• Proficiency in implementing monitoring, logging, and alerting solutions for data infrastructure (e.g., Prometheus, Grafana)

• Proficiency in distributed Linux environments