Title: Kafka Integration Platform Engineer

Summary & Responsibilities

As an Kafka Subject Matter Engineer you will manage the Kafka Elastic Kubernetes Service (EKS) on AWS integration platform. Your responsibilities will cover all areas of technical expertise as listed as experiences and qualifications below. Although you currently may not have all of those experiences, we expect you to be a willing and capable learner who stays up to date of the latest development within the area of responsibility

To succeed in this role you need to successfully communicate and collaborate with a range colleagues from the application and infra teams located in China as well as abroad,
Desirable experiences & qualifications

      1.    Kubernetes Expertise:

• Understanding of Kubernetes concepts like pods, deployments, services, and ingress.

• Proficiency in managing and troubleshooting Kubernetes clusters.

• Knowledge of Helm for managing Kubernetes applications.
      2.    AWS Services:

• Familiarity with AWS services, particularly EKS, EC2, VPC, IAM, and CloudWatch.

• Ability to configure and optimize AWS resources for Kafka, considering factors like storage, networking, and security

• Ability to use CloudFormation or Terraforms

      3.    Kafka Administration:

• Experience in Kafka setup, configuration, and performance tuning

• Knowledge of Kafka topics, partitions, replication, and consumer/producer configurations.

• Expertise in handling Kafka cluster failures and recovery.

      4.    Containerization and Docker:

• Proficiency in containerization using Docker, as EKS relies on Docker containers.

• Understanding of Docker images and how they interact with EKS.

      5.    Networking:

• Skills in managing network configurations within Kubernetes and AWS VPC

• Knowledge of how Kafka nodes communicate and ensuring secure communication.

• Ingress controller, ELB

      6.    Security:

• Expertise in securing Kafka clusters, including access controls and encryption.

• Implementing AWS IAM roles and policies for secure access.

• AAD, RBAC, Security groups

      7.    Monitoring and Logging:

• Familiarity with monitoring tools like Datadog and Cloudwatch.

• Setting up logging for Kafka and Kubernetes components, metrics, alerts logs.

      8.    Backup and Disaster Recovery:

• Implementation BCP and DR

• Planning and testing disaster recovery procedures.

      9.    Automation and Scripting:

• Ability to automate deployment and scaling processes using tools like Terraform, Ansible, or AWS CloudFormation.

      10.   Collaboration, Communication & Soft Skills:

• Effective communication skills for collaboration within the team and with other stakeholders

• Ability to document configurations, processes, and troubleshooting steps.

• Effective communication skills for collaboration with cross-functional teams located in China and abroad including gathering requirements, and articulating solutions..

• Ability to document configurations, processes, and troubleshooting steps.

• Teamwork: Ability to work collaboratively in a team environment and contribute to the success of projects.

• Adaptability: Willingness to learn new technologies, adapt to changing requirements, and explore innovative solutions.

• Time Management: Excellent time management skills to prioritize tasks, meet deadlines, and deliver high-quality results efficiently.

• Continuous learning and staying updated with the latest trends and advancements within your area.

      11.   GitLab:

• Ability to automate by CI/CD pipelines

• Infrastructure as code

• Monitor and test deployments


The position needs to travel and on-site customer site

其他信息
语言要求：英语
行业要求：全部行业