build RDS on top of Kubernetes, stateful workloads on k8s
主要是storage部分
Use non-local persistent block storage with XFS or ext4 to be more precise. Don’t use NFS. I warned you. Neither NFS v3 nor v4 will work. In short, a Kafka broker will terminate itself if it’s not able to delete a data directory due to the NFS “silly rename” problem. If you still don’t believe me then read this blog post very carefully. The storage has to be non-local so that Kubernetes is more flexible in choosing another node after restarts or relocations.
Headless Services: Services decouple the pods from clients through a logical name. Kubernetes takes care of load-balancing. However, with stateful workloads such as ZooKeeper and Kafka, clients have to communicate with a specific instance. This is where headless services come into play: as a client you still get a logical name but you don’t have to access the pod directly.
Helm is a package manager for Kubernetes comparable to OS package managers like yum, apt, Homebrew or Chocolatey.
The list of awesome operators mentions two operators for Kafka — one of them being Strimzi
Scaling a Kafka cluster is not an easy task. However, Kubernetes makes it very easy to scale pods to a certain number of replicas which means that the desired number of Kafka brokers can be defined declaratively. The hard part will be to reassign the partions after scaling up or before scaling down. Again, Kubernetes helps you with this task.