Openshift 3.11 OAuth认证配置手册

1. Oauth概述

Openshift 集群在安装的时候会内置一个oauth server，由它来完成oauth认证。client端发送业务请求，访问业务route的界面时候需要先发送认证请求到master，master上的oauth server接收认证请求，认证通过后，会将客户端的请求重定向到客户端应用需要访问的真实界面。

为了解决应用的安全性问题，比如某些应用有自己的图形界面，但是要访问的图形界面并没有任何的登陆认证机制，而这个应用的界面中能操作容器的某些高权限动作，如对pod的启停等操作。一旦某个未经授权或者认证的用户拿到了应用route的访问地址，即可登陆界面中，既有可能操作某些高危动作。为了避免这类问题的发生，红帽的openshift集群提供oauth认证方式，所有要访问应用界面的用户，点开应用界面route地址后，并不能直接访问应用界面，而是先跳转到openshift集群的登陆认证界面，需要用openshift用户登录，一旦登陆成功后，才跳转到应用界面中。

 \

2.OauthProxy

为了简化认证过程，社区提供了oauth proxy代理服务，用户只需要以sidecar的方式部署一个oauth_proxy在应用的pod中，就可以实现将外部访问业务系统界面时，要求下经过auth provider的认证，然后将流量经过proxy 路由到后端的应用中。如下图所示：

                           

上图中有两种配置方案，openshift中正常是有外部router的，理论来讲应该是左侧的配置方案，要求在route中配置这个证书加密；但是也可以直接选择openshift route 类型为passthrough，这样就能简化配置证书的次数，就采用了右侧示意图。只需要在oauth_proxy中配置TLS/SSL证书即可。即此证书是保证加密客户端访问到oauthproxy这一段通信的。

3        应用举例

3.1     Weave应用

Weave Scope 是 Docker 和 Kubernetes 可视化监控工具。提供了至上而下的集群基础设施和应用的完整视图，用户可以轻松对分布式的容器化应用进行实时监控和问题诊断。Weave Scope会自动生成容器之间的关系图，方便理解容器之间的关系，也方便监控容器化和微服务化的应用。尤其是在微服务应用大量存在的集群中，可以方便的观察微服务之间的关系，以及容器状态等信息。它会在openshift集群所有节点上部署一个deamonsetagent负责抓取数据，然后上报到weave scope应用，进行展示。

[root@master00 ~]# oc get ds

NAME                DESIRED   CURRENT  READY     UP-TO-DATE   AVAILABLE  NODE SELECTOR   AGE

weave-scope-agent   0        0         0         0            0           <none>          6d

安装完成后展示如下如所示，可以参看pod的关系图，和具体pod的列表等各种监控信息。同时可以按照project进行筛选过滤。但是这个界面最右侧可以看到，能对pod进行pause的操作。这是有风险的动作，所以一定要对此界面进行授权访问，openshift oauth认证就登场了。

 

3.2     部署weave

[root@master00 ~]# wget https://cloud.weave.works/k8s/scope.yaml

[root@master00 ~]# oc new-project weave-scope

Now using project "weave-scope"on server "https://master00.example.com:443".

 

You can add applications to this projectwith the 'new-app' command. For example, try:

 

   oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git

 

to build a new example application in Ruby.

[root@master00 ~]# oc create -f scope.yaml

Weave要求以root运行容器：

[root@master00 ~]# oc adm policyadd-cluster-role-to-user cluster-admin -z weave-scope

cluster role "cluster-admin"added: "weave-scope"

[root@master00 ~]# oc adm policyadd-scc-to-user privileged -z weave-scope

scc "privileged" added to:["system:serviceaccount:weave:weave-scope"]

[root@master00 ~]# oc adm policyadd-scc-to-user anyuid -z default

scc "anyuid" added to:["system:serviceaccount:weave:default"]

[root@master00 ~]# oc adm policyadd-scc-to-user anyuid -z weave-scope

scc "anyuid" added to:["system:serviceaccount:weave:weave-scope"]

[root@master00 ~]# oc expose svcweave-scope-app

[root@master00 ~]# oc get deployment

 

NAME                        DESIRED  CURRENT   UP-TO-DATE   AVAILABLE  AGE

weave-scope-app             1         1         1            1           12d

weave-scope-cluster-agent     1        1         1            1           12d

[root@ master00 ~]# oc get pod

NAME                                         READY     STATUS   RESTARTS   AGE

weave-scope-agent-25qbm                      1/1       Running  0          6d

weave-scope-agent-472w9                      1/1       Running  0          6d

weave-scope-agent-4zt47                      1/1       Running  0          6d

weave-scope-agent-64kxl                      1/1       Running  0          6d

weave-scope-agent-6tqfb                      1/1       Running  0          6d

weave-scope-agent-7hrhv                      1/1       Running  0          6d

weave-scope-agent-7lczd                      1/1       Running  0          6d

weave-scope-agent-8fl9d                      1/1       Running  0          6d

weave-scope-agent-d67m8                      1/1       Running  0          6d

weave-scope-agent-g4684                      1/1       Running  0          6d

weave-scope-agent-hzk2f                      1/1       Running  0          6d

weave-scope-agent-jvkt5                      1/1       Running  0          6d

weave-scope-agent-kt8h2                      1/1       Running  0          6d

weave-scope-agent-l2fcr                      1/1       Running  0          6d

weave-scope-agent-q8hk2                      1/1       Running  0          6d

weave-scope-agent-q9t7f                      1/1       Running  0          6d

weave-scope-agent-r4kwg                      1/1       Running  0          6d

weave-scope-agent-tfj4f                      1/1       Running  0          6d

weave-scope-agent-wcfr8                      1/1       Running  0          6d

weave-scope-agent-wk2zn                      1/1       Running  0          6d

weave-scope-agent-xjdmw                      1/1       Running  0          6d

weave-scope-app-85bc58bb5b-52zb5             2/2       Running  0          2h

weave-scope-cluster-agent-78d7ffdf85-zq7q4   1/1      Running   0          6d

[root@ master00 ~]# oc get svc

NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE

weave-scope-app   ClusterIP  172.30.134.142   <none>        80/TCP,8443/TCP   12d

 

[root@master00 ~]# oc get route

NAME                HOST/PORT PATH                            SERVICES             PORT     WILDCARD

weave-scope-app   weave-scope-app-weave.apps-27f9.generic.opentlc.com    weave-scope-app       app     None

3.3     配置oauth_proxy

• 修改weave 的deployment，添加如下红色部分内容

[root@master00 ~]# oc get deployment -oyaml weave-scope-app

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

 annotations:

   cloud.weave.works/launcher-info: |-

     {

       "original-request": {

         "url": "/k8s/v1.10/scope.yaml?k8s-version=1.11.0",

         "date": "Fri Sep 20 2019 01:54:18 GMT+0000 (UTC)"

       },

       "email-address": "[e-mail]"

     }

   deployment.kubernetes.io/revision: "45"

 creationTimestamp: 2019-12-04T06:09:54Z

 generation: 49

 labels:

   app: weave-scope

   name: weave-scope-app

   weave-cloud-component: scope

   weave-scope-component: app

 name: weave-scope-app

 namespace: weave-scope

 resourceVersion: "11017539"

 selfLink:/apis/extensions/v1beta1/namespaces/weave-scope/deployments/weave-scope-app

 uid: b0d6d95f-165c-11ea-a271-005056acd787

spec:

 progressDeadlineSeconds: 600

 replicas: 1

 revisionHistoryLimit: 2

 selector:

   matchLabels:

     app: weave-scope

     name: weave-scope-app

     weave-cloud-component: scope

     weave-scope-component: app

 strategy:

   rollingUpdate:

     maxSurge: 25%

     maxUnavailable: 25%

   type: RollingUpdate

 template:

   metadata:

     creationTimestamp: null

     labels:

       app: weave-scope

       name: weave-scope-app

       weave-cloud-component: scope

       weave-scope-component: app

   spec:

     containers:

     - args:

       - --mode=app

       command:

       - /home/weave/scope

       image: docker.io/weaveworks/scope:1.12.0

       imagePullPolicy: IfNotPresent

       name: app

       ports:

       - containerPort: 4040

         protocol: TCP

       resources: {}

       terminationMessagePath: /dev/termination-log

       terminationMessagePolicy: File

      - args:

        - --https-address=:8443

        - --provider=openshift

        - --upstream=http://localhost:4040

        - --tls-cert=/etc/tls/private/tls.crt

        - --tls-key=/etc/tls/private/tls.key

        - --cookie-secret=SECRET

        ---openshift-service-account=weave-scope

        ---openshift-sar={"namespace":"weave-scope","resource":"services","name":"weave-scope-app","verb":"get"}

        image: registry.example.com:5000/openshift/oauth-proxy:v3.11.153

        imagePullPolicy: IfNotPresent

        name: weave-scope-proxy

        resources: {}

        terminationMessagePath:/dev/termination-log

        terminationMessagePolicy: File

        volumeMounts:

        - mountPath: /etc/tls/private

          name: weave-scope-tls

     dnsPolicy: ClusterFirst

     restartPolicy: Always

     schedulerName: default-scheduler

     securityContext: {}

     serviceAccount: weave-scope

     serviceAccountName: weave-scope

     terminationGracePeriodSeconds: 30

      volumes:

      - name: weave-scope-tls

        secret:

          defaultMode: 420

          secretName: weave-scope-tls

status:

 availableReplicas: 1

 conditions:

  -lastTransitionTime: 2019-12-10T11:33:58Z

   lastUpdateTime: 2019-12-10T11:33:58Z

   message: Deployment has minimum availability.

   reason: MinimumReplicasAvailable

   status: "True"

   type: Available

  -lastTransitionTime: 2019-12-10T10:36:19Z

   lastUpdateTime: 2019-12-17T01:00:39Z

   message: ReplicaSet "weave-scope-app-85bc58bb5b" hassuccessfully progressed.

   reason: NewReplicaSetAvailable

   status: "True"

   type: Progressing

 observedGeneration: 49

 readyReplicas: 1

 replicas: 1

 updatedReplicas: 1

注意：

--https-address=:8443 指定的是oauth proxy监听的加密地址

--upstream=http://localhost:4040 指定的是weave的业务系统服务地址

--tls-cert=/etc/tls/private/tls.crt指定的是容器内加密前端访问8443端口的证书

--tls-key=/etc/tls/private/tls.key指定的是容器内加密前端访问8443端口的秘钥

证书的目录/etc/tls/private是用过volume weave-scope-tls 指定使用secret weave-scope-tls中的定义证书。

• 修改serviceaccount,添加如下红色部分

[root@master00 ~]# oc get sa

NAME          SECRETS   AGE

builder       2        12d

default       2        12d

deployer      2        12d

weave-scope   4        12d

[root@master00 ~]# oc get  sa weave-scope -o yaml

apiVersion: v1

imagePullSecrets:

- name: weave-scope-dockercfg-bnqkj

kind: ServiceAccount

metadata:

 annotations:

   cloud.weave.works/launcher-info: |-

     {

       "original-request": {

         "url": "/k8s/v1.10/scope.yaml?k8s-version=1.11.0",

          "date": "Fri Sep 20 201901:54:18 GMT+0000 (UTC)"

       },

       "email-address": "[e-mail]"

     }

   serviceaccounts.openshift.io/oauth-redirectreference.primary:'{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"weave-scope-app"}}'

 creationTimestamp: 2019-12-04T06:09:54Z

 labels:

   name: weave-scope

 name: weave-scope

 namespace: weave-scope

 resourceVersion: "11018043"

 selfLink: /api/v1/namespaces/weave-scope/serviceaccounts/weave-scope

 uid: b0d34368-165c-11ea-a271-005056acd787

secrets:

- name: weave-scope-token-w9lzl

- name: weave-scope-dockercfg-bnqkj

注意：

以上内容为1行，主要是使用serviceaccount作为oauth 的client ID，将访问weave-scope-app 这个route的流量使用serviceaccount作为的client ID和oauth server 进行认证。在deployment中也有指定serviceaccount的name.

• 修改service，添加如下红色部分

[root@master00 ~]# oc get svcweave-scope-app -o yaml

apiVersion: v1

kind: Service

metadata:

 annotations:

   cloud.weave.works/launcher-info: |-

     {

       "original-request": {

         "url":"/k8s/v1.10/scope.yaml?k8s-version=1.11.0",

         "date": "Fri Sep 20 2019 01:54:18 GMT+0000 (UTC)"

       },

       "email-address": "[e-mail]"

     }

   service.alpha.openshift.io/serving-cert-secret-name: weave-scope-tls

    service.alpha.openshift.io/serving-cert-signed-by:openshift-service-serving-signer@1573627524

 creationTimestamp: 2019-12-04T06:09:54Z

 labels:

   app: weave-scope

   name: weave-scope-app

   weave-cloud-component: scope

   weave-scope-component: app

 name: weave-scope-app

 namespace: weave-scope

 resourceVersion: "11018166"

 selfLink: /api/v1/namespaces/weave-scope/services/weave-scope-app

 uid: b0d90cd3-165c-11ea-a271-005056acd787

spec:

 clusterIP: 172.30.134.142

 ports:

  -name: app

   port: 80

   protocol: TCP

   targetPort: 4040

 - name: proxy

    port: 8443

    protocol: TCP

    targetPort: 8443

 selector:

   app: weave-scope

   name: weave-scope-app

   weave-cloud-component: scope

   weave-scope-component: app

 sessionAffinity: None

 type: ClusterIP

status:

 loadBalancer: {}

注意：

Port是service的端口

Targetport是oauth proxy的端口

 

• 修改route，更改如下红色部分

[root@master00 ~]# oc get routeweave-scope-app -o yaml

apiVersion: route.openshift.io/v1

kind: Route

metadata:

 annotations:

   openshift.io/host.generated: "true"

 creationTimestamp: 2019-12-10T11:36:58Z

 labels:

   app: weave-scope

   name: weave-scope-app

   weave-cloud-component: scope

   weave-scope-component: app

 name: weave-scope-app

 namespace: weave-scope

 resourceVersion: "11027677"

 selfLink:/apis/route.openshift.io/v1/namespaces/weave-scope/routes/weave-scope-app

 uid: 602a788f-1b41-11ea-9ece-005056acf6c9

spec:

 host: weave-scope-app-weave.apps-27f9.generic.opentlc.com

 port:

    targetPort: proxy

 tls:

    insecureEdgeTerminationPolicy: Redirect

    termination: passthrough

  to:

   kind: Service

   name: weave-scope-app

   weight: 100

 wildcardPolicy: None

status:

 ingress:

  -conditions:

    -lastTransitionTime: 2019-12-17T00:55:17Z

     status: "True"

     type: Admitted

   host: weave-scope-app-weave.apps-27f9.generic.opentlc.com

   routerName: router

   wildcardPolicy: None

• 创建secret

[root@master00 ~]# oc create secret tlsweave-scope-tls --cert=/etc/origin/master/ca.crt--key=/etc/origin/master/ca.key

[root@master00 ~]# oc get secret

NAME                          TYPE                                  DATA      AGE

builder-dockercfg-4jvpr       kubernetes.io/dockercfg               1         12d

builder-token-gmsv6          kubernetes.io/service-account-token  4         12d

builder-token-xt5nt          kubernetes.io/service-account-token  4         12d

default-dockercfg-nbkzq       kubernetes.io/dockercfg               1         12d

default-token-hp8df           kubernetes.io/service-account-token   4        12d

default-token-nknb4          kubernetes.io/service-account-token  4         12d

deployer-dockercfg-x4v9j      kubernetes.io/dockercfg               1         12d

deployer-token-4dksk          kubernetes.io/service-account-token   4        12d

deployer-token-v2crb         kubernetes.io/service-account-token  4         12d

weave-scope-dockercfg-bnqkj   kubernetes.io/dockercfg               1         12d

weave-scope-tls               kubernetes.io/tls                     2         3h

weave-scope-token-nm298       kubernetes.io/service-account-token   4        12d

weave-scope-token-w9lzl       kubernetes.io/service-account-token   4        12d

至此，全部配置完毕。

重启weave-scope-apppod，访问route地址。