Job Title

Kafka Expert Data Streaming Engineer We are seeking a highly skilled Kafka Expert with deep expertise in Apache Kafka, Linux systems, and Kubernetes to join our data platform team. This is a hands-on engineering role focused on building and optimizing real-time data streaming solutions that are scalable, secure, and production-ready. Youll work closely with infrastructure and development teams to design streaming architectures, enable robust use cases, and ensure high availability and performance across distributed systems. Key Responsibilities Design, deploy, and manage Apache Kafka clusters in development, testing, and production. Optimize Kafka performance, reliability, and scalability for high-throughput pipelines. Integrate Kafka with tools such as Kafka Connect, Schema Registry, MirrorMaker, and Kafka Streams . Deploy and operate Kafka on Kubernetes using Helm, Operators, or custom manifests. Manage and troubleshoot Linux systems (Red Hat, Debian, Ubuntu) supporting Kafka. Build automation and Infrastructure-as-Code (IaC) using Ansible, Terraform, GitLab CI/CD, and GitOps practices. Implement observability and alerting with Prometheus, Grafana, Fluent Bit, Spark UI, and Flink Dashboard . Lead incident response and root cause analysis for Kafka and related systems. Collaborate with teams to build end-to-end data pipelines for real-time integration, event-driven microservices, and logging/monitoring. Design and operate Spark Structured Streaming and Flink DataStream pipelines , including: Resource allocation, scaling, and job scheduling. State management, checkpointing, and fault tolerance. Performance tuning for low latency and high throughput. Required Qualifications 5+ years administering and supporting Apache Kafka in production. Strong Linux system administration skills (Red Hat, Debian, Ubuntu). Solid experience with Kubernetes (OpenShift, Rancher, or upstream distributions). Proficiency in scripting and automation (Bash, Python, Ansible). Experience with IaC, CI/CD pipelines, and GitOps . Strong background in Kafka security, monitoring, and schema management . 4+ years building scalable pipelines with Apache Spark and/or Apache Flink , with expertise in performance tuning and distributed systems. #J-18808-Ljbffr