Cynet Systems
Senior Administrator- Cloudera Big Data
Cynet Systems, Reston, Virginia, United States, 22090
Job Description:
Additional Must Have Skills Includes:
Cloudera CDP Public Cloud v7.2.17 or higher. pache Kafka - strong Administration & troubleshooting skills. Kafka Streams API. stream processing with KStreams & Ktables. Kafka integration with IBM MQ. Kafka broker management. Topic/ offset management. pache Nifi - Administration. Flow management. registry server management. controller service management. NiFi to Kafka /HBase /SOLR integration. Hbase - administration. database management. troubleshooting. SOLR - administration. managing Logging level. managing shards & high availability. Collection management. Rectify resource intensive & long running SOLR queries. Proficient with handling AWS EC2, S3, EBS, EFS. Ensure Cloudera installation and configuration is at optimal specifications (CDP, CDSW, Hive, Spark, NiFi). Design and implement big data pipelines and automated data flows using Python/R and NiFi. ssist and provide expertise as it pertains to automating the entire project lifecycle. Perform incremental updates and upgrades to the Cloudera environment with newer versions. ssist with new use cases (i.e., analytics/ML, data science, data ingest and processing), Infrastructure (including new cluster deployments, cluster migration, expansion, major upgrades, COOP/DR, and security). ssist in testing, governance, data quality, training, and documentation efforts. Move data and use YARN to allocate resources and schedule jobs. Manage job workflows with Hue. Implement comprehensive security policies across the Hadoop cluster using Ranger. Troubleshoot potential issues with Kerberos, TLS/SSL, Models, and Experiments, as well as other workload issues that data scientists might encounter once the application is running. Supporting the Big Data / Hadoop databases throughout the development and production lifecycle. Troubleshooting and resolving database integrity issues, performance issues, blocking and deadlocking issues, replication issues, log shipping issues, connectivity issues, security issues, performance tuning, query optimization, using monitoring and troubleshooting tools. Create, test, and implement scripting for automation support. Experience in working with Kafka ecosystem (Kafka Brokers, Connect, Zookeeper) in production is ideal. Implement and support streaming technologies such as Kafka, Spark & Kudu.
Additional Must Have Skills Includes:
Cloudera CDP Public Cloud v7.2.17 or higher. pache Kafka - strong Administration & troubleshooting skills. Kafka Streams API. stream processing with KStreams & Ktables. Kafka integration with IBM MQ. Kafka broker management. Topic/ offset management. pache Nifi - Administration. Flow management. registry server management. controller service management. NiFi to Kafka /HBase /SOLR integration. Hbase - administration. database management. troubleshooting. SOLR - administration. managing Logging level. managing shards & high availability. Collection management. Rectify resource intensive & long running SOLR queries. Proficient with handling AWS EC2, S3, EBS, EFS. Ensure Cloudera installation and configuration is at optimal specifications (CDP, CDSW, Hive, Spark, NiFi). Design and implement big data pipelines and automated data flows using Python/R and NiFi. ssist and provide expertise as it pertains to automating the entire project lifecycle. Perform incremental updates and upgrades to the Cloudera environment with newer versions. ssist with new use cases (i.e., analytics/ML, data science, data ingest and processing), Infrastructure (including new cluster deployments, cluster migration, expansion, major upgrades, COOP/DR, and security). ssist in testing, governance, data quality, training, and documentation efforts. Move data and use YARN to allocate resources and schedule jobs. Manage job workflows with Hue. Implement comprehensive security policies across the Hadoop cluster using Ranger. Troubleshoot potential issues with Kerberos, TLS/SSL, Models, and Experiments, as well as other workload issues that data scientists might encounter once the application is running. Supporting the Big Data / Hadoop databases throughout the development and production lifecycle. Troubleshooting and resolving database integrity issues, performance issues, blocking and deadlocking issues, replication issues, log shipping issues, connectivity issues, security issues, performance tuning, query optimization, using monitoring and troubleshooting tools. Create, test, and implement scripting for automation support. Experience in working with Kafka ecosystem (Kafka Brokers, Connect, Zookeeper) in production is ideal. Implement and support streaming technologies such as Kafka, Spark & Kudu.