请问在cloud上zookeeper的ip怎么设置

来源:7-4 【不熟悉Dubbo的筒子看过来】Dubbo快速入门

再也不上QQ鸟

2020-06-19

我想在google kubernetes engine上想部署java web服务,请问在cloud上zookeeper的ip怎么设置

用docker起zookeeper服务,在本地环境java web服务是没问题的,但是一旦在k8s上运行后,java web所运行的pod就无法连接docker上的zookeeper

注册完zookeeper
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b99c41f81335 zookeeper:3.5 “/docker-entrypoint.…” 8 minutes ago Up 8 minutes 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp nostalgic_nash

查看zookeeper的ip address
$ docker inspect b99c4
"Networks": {
“bridge”: {
“IPAMConfig”: null,
“Links”: null,
“Aliases”: null,
“NetworkID”: “a03e6772fc858a1e14847f0d2a8edcb2d0ee39ddb2449f5f62b89da73d54ce51”,
“EndpointID”: “53d85011cb72cd74dd8d7bd6c469a5f6416c11ec959ad22f86ad9904a9c1ded8”,
“Gateway”: “172.18.0.1”,
“IPAddress”: “172.18.0.2”,
“IPPrefixLen”: 16,
“IPv6Gateway”: “”,
“GlobalIPv6Address”: “”,
“GlobalIPv6PrefixLen”: 0,
“MacAddress”: “02:42:ac:12:00:02”,
“DriverOpts”: null
}
}
配置好web服务的zookeeper的ip
cat ./WEB-INF/classes/applicationContext-service-config.xml

<?xml version="1.0" encoding="UTF-8"?>

    <dubbo:application name="demo" />
    <dubbo:registry address="zookeeper://172.18.0.2:2181" />
    <dubbo:consumer retries="3" timeout="60000" />

用docker启动web服务
docker run -it gcr.io/lithe-sonar-276007/web:v1

ete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
20-Jun-2020 01:32:40.931 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/ROOT] has finished in [8,467] ms
20-Jun-2020 01:32:40.954 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [“http-nio-8080”]
20-Jun-2020 01:32:41.065 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [8,781] milliseconds

查看服务启动成功
tank31xj2000@cloudshell:~/mooc-k8s-demo/web-demo (lithe-sonar-276007)$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2c051fa92296 gcr.io/lithe-sonar-276007/web:v1 “sh /usr/local/tomca…” 9 minutes ago Up 9 minutes 8080/tcp reverent_mahavira
b99c41f81335 zookeeper:3.5 “/docker-entrypoint.…” 29 minutes ago Up 29 minutes 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp nostalgic_nash

tank31xj2000@cloudshell:~/mooc-k8s-demo/web-demo (lithe-sonar-276007)$ docker exec -it 2c051fa92296 sh

Hello tank31#

编写start.sh
tank31xj2000@cloudshell:~/mooc-k8s-demo/web-demo (lithe-sonar-276007)$ cat start.sh
#!/bin/bash

sh /usr/local/tomcat/bin/startup.sh

tail -f /usr/local/tomcat/logs/catalina.out

按照google kubernetes engine编写web.yaml,去掉了host,加上了 annotations
tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$cat web.yaml
#deploy
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-demo
spec:
selector:
matchLabels:
app: web-demo
replicas: 1
template:
metadata:
labels:
app: web-demo
spec:
containers:
- name: web-demo
image: gcr.io/lithe-sonar-276007/web:v1
ports:
- containerPort: 8080

#service
apiVersion: v1
kind: Service
metadata:
name: web-demo
spec:
ports:

  • port: 80
    protocol: TCP
    targetPort: 8080
    selector:
    app: web-demo
    type: ClusterIP

#ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: web-demo
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:

  • http:
    paths:
    • path: /
      backend:
      serviceName: web-demo
      servicePort: 80

根据yaml启动服务
tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ kubectl apply -f web.yaml
deployment.apps/web-demo created
service/web-demo created
ingress.extensions/web-demo created
tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ kubectl get pods
NAME READY STATUS RESTARTS AGE
my-release-jenkins-99fcfd99-7xlrc 2/2 Running 0 7d19h
nginx-ingress-controller-648668ff47-hxp9n 1/1 Running 0 8d
nginx-ingress-default-backend-7db6cc5bf-gdp8r 1/1 Running 0 8d
web-demo-6bc775d749-xpld6 1/1 Running 0 53s
tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.0.0.1 443/TCP 18d
nginx-ingress-controller LoadBalancer 10.0.3.56 35.194.172.175 80:30853/TCP,443:32663/TCP 8d
nginx-ingress-default-backend ClusterIP 10.0.11.81 80/TCP 8d
web-demo ClusterIP 10.0.12.218 80/TCP 60s

查看web服务log
tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ kubectl exec -it web-demo-6bc775d749-xpld6 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl kubectl exec [POD] – [COMMAND] instead.

cd logs

ls

catalina.2020-06-20.log catalina.out host-manager.2020-06-20.log localhost.2020-06-20.log localhost_access_log.2020-06-20.txt manager.2020-06-20.log

cat catalina.out

ain] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/ROOT]
20-Jun-2020 02:34:29.432 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
[2020-06-20 02:34:37.639] {ERROR} org.apache.curator.ConnectionState 228 - Connection timed out for connection string (172.18.0.2:2181) and timeout (5000) / elapsed (5209)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225)
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94)

写回答

2回答

再也不上QQ鸟

提问者

2020-06-20

我将zookeeper改成host模式

tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ docker container ls

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES

e6e91ffd304d        zookeeper:3.5       "/docker-entrypoint.…"   2 hours ago         Up 2 hours                              zookeeper3.5

tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ docker inspect e6e9 | grep Network

            "NetworkMode": "host",

        "NetworkSettings": {

            "Networks": {

                    "NetworkID": "8e74669fcc82366f7b4bc933c8854c558d59179bf34ffa297c58d48f3d5f8601",


web服务也改成host模式部署

#deploy

apiVersion: apps/v1

kind: Deployment

metadata:

  name: web-demo

spec:

  selector:

    matchLabels:

      app: web-demo

  replicas: 1

  template:

    metadata:

      labels:

        app: web-demo

    spec:

      containers:

      - name: web-demo

        image: gcr.io/lithe-sonar-276007/web:v2

        ports:

        - containerPort: 8080

      hostNetwork: true



但是启动的pod和宿主机的ip不在一个网段上,依然没法访问zookeeper

Pod 的IP


tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ kubectl describe pod  web-demo-6c4dc668d5-bqs84 | grep IP

IP:             10.140.0.5

IPs:            <none>


宿主机的IP

tank31xj2000@cloudshell:~/mooc-k8s-demo/configs (lithe-sonar-276007)$ ip a | grep inet

    inet 127.0.0.1/8 scope host lo

    inet 172.18.0.1/16 brd 172.18.255.255 scope global docker0

    inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0


请问如何让pod访问宿主机的服务


0
1
刘果国
感觉这个问题有点复杂化了,咱们捋捋,首先podip跟主机ip不同网段是符合预期的,k8s的cni约定保证了podip可以访问任何宿主机可以访问的网络。从你发的内容看貌似这个网络不通,所以先检查从pod里访问任意一台worker节点的ip,是否能ping通,如果这块就有问题那底层的cni网络肯定就有问题了。
2020-06-21
共1条回复

刘果国

2020-06-20

从这个配置看

“apiVersion: v1
kind: Service
metadata:
 name: zk-cs
 labels:
   app: zk
spec:
 ports:
 - port: 2181
   name: client
 selector:
   app: zk”

这个服务暴露方式是集群内通过dns访问。外部网络无法直接访问。

如果直接docker run启动的话,可以使用host网络或者桥接网络+端口映射把服务暴露在宿主机上。

0
1
再也不上QQ鸟
不好意思,修改了一下问题内容,把docker run启动的过程记录下来了
2020-06-20
共1条回复

Kubernetes生产落地全程实践

一个互联网公司落地Kubernetes全过程点点滴滴

2293 学习 · 2216 问题

查看课程