calico默认使用IPIP网络模型，要修改将其改为BGP网格模型需要修改其地址池的详细配置，BGP网格模式一般适用于小规模的网络。

BGP网格模式启用方法

1.获取地址池详细配置导出

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
root@k8s-master01:~# kubectl calico get ippools -o yaml
apiVersion: projectcalico.org/v3
items:
- apiVersion: projectcalico.org/v3
  kind: IPPool
  metadata:
    creationTimestamp: "2021-08-06T06:00:24Z"
    name: default-ipv4-ippool
    resourceVersion: "6789"
    uid: 943b85b2-9759-49ce-8f73-78f1f3f8a111
  spec:
    blockSize: 24
    cidr: 192.168.0.0/16
    ipipMode: Always
    natOutgoing: true
    nodeSelector: all()
    vxlanMode: Never
kind: IPPoolList
metadata:
  resourceVersion: "9306"

# 导出为资源清单
root@k8s-master01:~/yaml/chapter10# kubectl calico get ippools -o yaml > default-ipv4-ippool.yaml

2.对导出的配置清单进行修改

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
root@k8s-master01:~/yaml/chapter10# vim default-ipv4-ippool.yaml
apiVersion: projectcalico.org/v3
items:
- apiVersion: projectcalico.org/v3
  kind: IPPool
  metadata:
    creationTimestamp: "2021-08-06T06:00:24Z"
    name: default-ipv4-ippool
    resourceVersion: "6789"
    uid: 943b85b2-9759-49ce-8f73-78f1f3f8a111
  spec:
    blockSize: 24
    cidr: 192.168.0.0/16
    ipipMode: CrossSubnet      # 将ipipMode改为CrossSubnet或Never
    natOutgoing: true
    nodeSelector: all()
    vxlanMode: Never
kind: IPPoolList
metadata:
  resourceVersion: "9418"

# 改BGP模式需要修改ipipMode，其有2各选项可以改为BGP
# CrossSubnet表示混杂模式也就是混合模式，表示跨节点子网时才使用IPIP
# Never表示纯BGP模式
# vxlanMode: CrossSubnet ipipMode: Never 表示VxLan的混合模型

3.将其重新应用到网络中

1
2
root@k8s-master01:~/yaml/chapter10# kubectl calico apply -f default-ipv4-ippool.yaml
Successfully applied 1 'IPPool' resource(s)

4.BGP生效后再次查看路由信息

1
2
3
4
5
6
7
8
9
10
root@k8s-master01:~/yaml/chapter10# ip route list
default via 172.16.11.1 dev eth0 proto static
172.16.11.0/24 dev eth0 proto kernel scope link src 172.16.11.71
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.30.0/24 via 172.16.11.83 dev eth0 proto bird
192.168.96.0/24 via 172.16.11.82 dev eth0 proto bird
blackhole 192.168.130.0/24 proto bird
192.168.130.1 dev cali1dceed2d547 scope link
192.168.130.2 dev calif39be49e967 scope link
192.168.131.0/24 via 172.16.11.81 dev eth0 proto bird

可以看出当前的路由信息类似于flannel的host-gw，不再经由tunl0，而是直接由物理网卡eth0接口到bird向外发送。

抓包验证

1.部署deployment资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
root@k8s-master01:~/yaml/chapter08# VERSION=v1.0 envsubst < deployment-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-demo
spec:
  replicas: 4
  selector:
    matchLabels:
      app: demoapp
      release: stable
  template:
    metadata:
      labels:
        app: demoapp
        release: stable
    spec:
      containers:
      - name: demoapp
        image: ikubernetes/demoapp:v1.0
        ports:
        - containerPort: 80
          name: http

root@k8s-master01:~/yaml/chapter08# VERSION=v1.0 envsubst < deployment-demo.yaml | kubectl apply -f -
deployment.apps/deployment-demo created

# 创建出4个Pod
root@k8s-master01:~/yaml/chapter08# kubectl get pods -o wide
NAME                              READY   STATUS    RESTARTS   AGE   IP              NODE         NOMINATED NODE   READINESS GATES
deployment-demo-fb544c5d8-8jd62   1/1     Running   0          19s   192.168.96.1    k8s-node02   <none>           <none>
deployment-demo-fb544c5d8-wcdms   1/1     Running   0          19s   192.168.131.3   k8s-node01   <none>           <none>
deployment-demo-fb544c5d8-xvsm7   1/1     Running   0          19s   192.168.30.2    k8s-node03   <none>           <none>
deployment-demo-fb544c5d8-zwbhd   1/1     Running   0          19s   192.168.30.1    k8s-node03   <none>           <none>

2.在节点1上进行抓包

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# 使用node01上的pod访问node03上pod
root@k8s-master01:~/yaml/chapter08# kubectl exec deployment-demo-fb544c5d8-xvsm7 -- curl 192.168.131.3
iKubernetes demoapp v1.0 !! ClientIP: 192.168.30.2, ServerName: deployment-demo-fb544c5d8-wcdms, ServerIP: 192.168.131.3!

# 同时在node01上进行抓包
root@k8s-node01:~# tcpdump -i eth0 -nn ip host 192.168.30.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
07:41:00.255112 IP 192.168.30.2.53310 > 192.168.131.3.80: Flags [S], seq 609498676, win 43200, options [mss 1440,sackOK,TS val 3721746601 ecr 0,nop,wscale 9], length 0
07:41:00.255246 IP 192.168.131.3.80 > 192.168.30.2.53310: Flags [S.], seq 2360321282, ack 609498677, win 42840, options [mss 1440,sackOK,TS val 389171798 ecr 3721746601,nop,wscale 9], length 0
07:41:00.255512 IP 192.168.30.2.53310 > 192.168.131.3.80: Flags [.], ack 1, win 85, options [nop,nop,TS val 3721746602 ecr 389171798], length 0
07:41:00.255578 IP 192.168.30.2.53310 > 192.168.131.3.80: Flags [P.], seq 1:78, ack 1, win 85, options [nop,nop,TS val 3721746602 ecr 389171798], length 77: HTTP: GET / HTTP/1.1
07:41:00.255613 IP 192.168.131.3.80 > 192.168.30.2.53310: Flags [.], ack 78, win 84, options [nop,nop,TS val 389171798 ecr 3721746602], length 0
07:41:00.257483 IP 192.168.131.3.80 > 192.168.30.2.53310: Flags [P.], seq 1:18, ack 78, win 84, options [nop,nop,TS val 389171800 ecr 3721746602], length 17: HTTP: HTTP/1.0 200 OK
07:41:00.257863 IP 192.168.131.3.80 > 192.168.30.2.53310: Flags [FP.], seq 18:277, ack 78, win 84, options [nop,nop,TS val 389171801 ecr 3721746602], length 259: HTTP
07:41:00.258108 IP 192.168.30.2.53310 > 192.168.131.3.80: Flags [.], ack 18, win 85, options [nop,nop,TS val 3721746604 ecr 389171800], length 0
07:41:00.259058 IP 192.168.30.2.53310 > 192.168.131.3.80: Flags [F.], seq 78, ack 278, win 85, options [nop,nop,TS val 3721746605 ecr 389171801], length 0
07:41:00.259122 IP 192.168.131.3.80 > 192.168.30.2.53310: Flags [.], ack 79, win 84, options [nop,nop,TS val 389171802 ecr 3721746605], length 0

# 可以看出 192.168.20.2与192.168.131.3直接进行通信没有内层报文存在了

其他

1.查看node状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
root@k8s-master01:~/yaml/chapter08# kubectl-calico node status
Calico process is running.

IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| PEER ADDRESS |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+--------------+-------------------+-------+----------+-------------+
| 172.16.11.81 | node-to-node mesh | up    | 06:08:26 | Established |
| 172.16.11.82 | node-to-node mesh | up    | 06:08:27 | Established |
| 172.16.11.83 | node-to-node mesh | up    | 06:08:27 | Established |
+--------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

# 由于当前是工作在BGP的peer模型下，此处是一个一对一的模型，n对n-1的mesh网络