近期在学习AWS EKS Fargate服务,也打算在AWS上部署EKS Fargete服务,在部署过程中发现,只有完整的portal图形化部署,或者eksctl这种打包的部署方式。为了寻求EKS Fargate部署细节,打算使用CLI的方式进行部署,笔者认为这样才能更多的了解云服务的后台运行机制。在Winston老师的帮助下,完成这个部署,分享出来,希望对大家有所帮助

0. 前期准备

0.1. 准备VPC

因为AWS EKS Fargate只能部署在关联NAT Gateway的的private Subnet中,建议准备包含公有子网和私有的子网的VPC,本例中准备如下网络环境

EKS_VPC copy.drawio.png

0.2. 准备安全组

创建安全组

aws ec2
create-security-group --group-name ControlPlaneSecurityGroup --description
"ControlPlaneSecurityGroup" --vpc-id vpc-0259b9606617599cf

标记安全组

aws
ec2 create-tags --resources sg-00ce55bd1fca82f69 --tags
Key=Name,Value="ControlPlaneSecurityGroup"

记录下安全组ID,sg-00ce55bd1fca82f69,以备后面使用

1. 部署EKS Master

1.1 创建IAM role for EKS

设置策略文档

cat > cluster-role-trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

创建角色,角色名为<b>GaryAmazonEKSClusterRole</b>

aws iam create-role \
     --role-name GaryAmazonEKSClusterRole  \
     --assume-role-policy-document file://"cluster-role-trust-policy.json"

将所需的 Amazon EKS 托管 IAM 策略附加到角色

aws iam attach-role-policy \
    --policy-arn arn:aws:iam::aws:policy/AmazonEKSClusterPolicy \
    --role-name GaryAmazonEKSClusterRole

1.2 创建 EKS Master

创建EKS,参数如下:

region为:宁夏区(ap-northeast-1)

EKS集群名为:<b>GaryEKS</b>

EKS集群版本为:1.21

角色ARN:arn:aws:iam::425052328337:role/GaryAmazonEKSClusterRole

SubnetID为subnet-004187a55e129319a,subnet-0c530dcf64959a1f6

aws eks create-cluster \
   --region ap-northeast-1 \
   --name GaryEKS  \
   --kubernetes-version 1.21 \
   --role-arn arn:aws:iam::425052328337:role/GaryAmazonEKSClusterRole \
   --resources-vpc-config subnetIds=subnet-004187a55e129319a,subnet-0c530dcf64959a1f6,securityGroupIds=sg-00ce55bd1fca82f69

查询EKS部署状态

aws eks describe-cluster \
    --region ap-northeast-1 \
    --name GaryEKS \
    --query "cluster.status"

2. 部署EKS Fargate Woker Node角色

2.1 创建Node角色

创建Node节点角色策略

cat > pod-execution-role-trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks-fargate-pods.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

创建Node角色

aws iam create-role \
  --role-name GaryAmazonEKSFargatePodExecutionRole \
  --assume-role-policy-document file://"pod-execution-role-trust-policy.json"

附加策略

aws iam attach-role-policy \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy \
  --role-name GaryAmazonEKSFargatePodExecutionRole

3. 更新Coredns

3.1 部署coredns profile

aws eks create-fargate-profile \
    --fargate-profile-name coredns \
    --cluster-name GaryEKS \
    --pod-execution-role-arn arn:aws:iam::425052328337:role/GaryAmazonEKSFargatePodExecutionRole \
    --selectors namespace=kube-system,labels={k8s-app=kube-dns} \
    --subnets subnet-004187a55e129319a subnet-0c530dcf64959a1f6

部署coredns patch

kubectl patch deployment coredns \
   -n kube-system \
   --type json \
   -p='[{"op": "remove", "path": "/spec/template/metadata/annotations/eks.amazonaws.com~1compute-type"}]'

3.2 配置命名空间

配置命名空间yaml文件

cat > aws-observability-namespace.yaml <<EOF
kind: Namespace
apiVersion: v1
metadata:
  name: aws-observability
  labels:
    aws-observability: enabled
EOF

创建命名空间

kubectl apply -f aws-observability-namespace.yaml

创建命名空间,并将metric发送到Cloudwatch

cat >  aws-logging-cloudwatch-configmap.yaml << EOF
kind: ConfigMap
apiVersion: v1
metadata:
  name: aws-logging
  namespace: aws-observability
data:
  output.conf: |
    [OUTPUT]
        Name cloudwatch_logs
        Match   *
        region ap-northeast-1
        log_group_name fluent-bit-cloudwatch
        log_stream_prefix from-fluent-bit-
        auto_create_group true

  parsers.conf: |
    [PARSER]
        Name crio
        Format Regex
        Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>P|F) (?<log>.*)$
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L%z

  filters.conf: |
     [FILTER]
        Name parser
        Match *
        Key_name log
        Parser crio
        Reserve_Data On
        Preserve_Key On
EOF

将配置清单应用到集群

kubectl apply -f aws-logging-cloudwatch-configmap.yaml

将cloudwatch IAM策略下载

curl -o permissions.json https://raw.githubusercontent.com/aws-samples/amazon-eks-fluent-logging-examples/mainline/examples/fargate/cloudwatchlogs/permissions.json

创建IAM策略

aws iam create-policy --policy-name eks-fargate-logging-policy --policy-document file://permissions.json

将 IAM 策略附加到为 Fargate 配置文件指定的 Pod 执行角色

aws iam attach-role-policy \
  --policy-arn arn:aws:iam::425052328337:policy/eks-fargate-logging-policy \
  --role-name GaryAmazonEKSFargatePodExecutionRole

重启coredns服务

kubectl rollout restart -n kube-system deployment coredns

4. 创建default命名空间

创建EKS Fargateprofile

aws eks create-fargate-profile \
  --fargate-profile-name Garyfargate-default \
  --cluster-name GaryEKS \
  --pod-execution-role-arn arn:aws:iam::425052328337:role/GaryAmazonEKSFargatePodExecutionRole \
--selectors namespace=default