当前位置:   article > 正文

安装kubernetes_如何在Kubernetes中安装和使用Linkerd

linkerd+kubernetes

安装kubernetes

The author selected the Tech Education Fund to receive a donation as part of the Write for DOnations program.

作者选择了Tech Education Fund作为“ Write for DOnations”计划的一部分来接受捐赠。

介绍 (Introduction)

A service mesh is a dedicated infrastructure layer that helps administrators handle service-to-service communication. Offering many powerful tools, these service meshes can make your system safer, more reliable, and more visible too.

服务网格是专用的基础结构层,可帮助管理员处理服务到服务的通信。 这些服务网格提供了许多强大的工具,可以使您的系统更安全,更可靠,更可见。

A service mesh like Linkerd, for example, can automatically encrypt connections, handle request retries and timeouts, provide telemetry information like success rates and latencies, and more.

例如, Linkerd之类的服务网格可以自动加密连接,处理请求重试和超时,提供遥测信息(如成功率和等待时间)等。

In this tutorial, you will install the Linkerd service mesh in your Kubernetes cluster, deploy an example application, and then explore Linkerd’s dashboard. After familiarizing yourself with some of this dashboard information, you will configure Linkerd to enforce timeout and retry policies for a particular Kubernetes pod.

在本教程中,您将在Kubernetes集群中安装Linkerd服务网格,部署示例应用程序,然后浏览Linkerd的仪表板。 在熟悉了一些仪表板信息之后,您将配置Linkerd以对特定的Kubernetes容器强制执行timeout并重retry策略。

Alternately, consider exploring DigitalOcean’s one-click Linkerd/Kubernetes installation option.

或者,考虑浏览DigitalOcean的一键式Linkerd / Kubernetes安装选项

先决条件 (Prerequisites)

步骤1 —部署应用程序 (Step 1 — Deploying the Application)

To see Linkerd in action, you need to have an application running in your cluster. In this step you will deploy an application called emojivoto, which the Linkerd team created for this purpose.

要查看运行中的Linkerd,您需要在集群中运行一个应用程序。 在这一步中,您将部署一个名为emojivoto的应用程序, Linkerd团队为此目的创建了该应用程序。

In this repository, you can see the code for the four services that compose the application, as well as the manifest file that you will use to deploy these services in your Kubernetes cluster.

该存储库中 ,您可以看到组成应用程序的四个服务的代码,以及将用于在Kubernetes集群中部署这些服务的清单文件

First, save this manifest file locally:

首先,将此清单文件保存在本地:

  • curl https://run.linkerd.io/emojivoto.yml --output manifest.yaml

    curl https://run.linkerd.io/emojivoto.yml-输出清单.yaml

You are using curl to fetch the file, then passing the --output option to tell it where you want the file saved. In this case, you are creating a file called manifest.yaml.

您正在使用curl来获取文件,然后传递--output选项来告诉它要将文件保存在何处。 在这种情况下,您将创建一个名为manifest.yaml的文件。

To better understand what this file will accomplish, inspect its contents with cat or open it with your favorite editor:

为了更好地了解此文件将完成的工作,请使用cat检查其内容或使用您喜欢的编辑器将其打开:

  • cat manifest.yaml | less

    猫manifest.yaml | 减

Press SPACE to page through the directives. You will see that manifest.yaml is creating a Kubernetes namespace called emojivoto where everything related to this application will run, and a couple of Kubernetes Deployments and Services.

SPACE键以翻阅指令。 您将看到manifest.yaml正在创建一个名为emojivoto的Kubernetes命名空间,以及与该应用程序相关的所有内容都将在其中运行,以及几个Kubernetes DeploymentsServices

Next, apply this manifest in your Kubernetes cluster:

接下来,将此清单应用于您的Kubernetes集群:

  • kubectl apply -f manifest.yaml

    kubectl套用-f manifest.yaml

Again, you’re using kubectl apply with the -f flag to assign a file that you want to apply.

同样,您使用带有-f标志的kubectl apply来分配要应用的文件。

This command will output a list of all the resources that were created:

此命令将输出所有已创建资源的列表:

  1. Output
  2. namespace/emojivoto created
  3. serviceaccount/emoji created
  4. serviceaccount/voting created
  5. serviceaccount/web created
  6. service/emoji-svc created
  7. service/voting-svc created
  8. service/web-svc created
  9. deployment.apps/emoji created
  10. deployment.apps/vote-bot created
  11. deployment.apps/voting created
  12. deployment.apps/web created

Now check that the services are running:

现在检查服务是否正在运行:

  • kubectl -n emojivoto get pods

    kubectl -n emojivoto获得豆荚

You are using kubectl to list all the pods you have running in your cluster, and then passing the -n flag to indicate which namespaces you want to use. You’re passing the emojivoto namespace because that is where you are running all these services.

您正在使用kubectl列出集群中正在运行的所有pods ,然后传递-n标志以指示要使用的命名空间。 您正在传递emojivoto命名空间,因为这是您运行所有这些服务的地方。

When you see all the pods in the Running state, you are good to go:

当您看到所有pods处于“ Running状态时,您就可以开始:

  1. Output
  2. NAME READY STATUS RESTARTS AGE
  3. emoji-566954596f-cw75b 1/1 Running 0 24s
  4. vote-bot-85c5f5699f-7dw5c 1/1 Running 0 24s
  5. voting-756995b6fc-czf8z 1/1 Running 0 24s
  6. web-7f7b69d467-2546n 1/1 Running 0 23s

Finally, to see the application running in your browser, you will use the kubectl built-in feature of forwarding local requests to your remote cluster:

最后,要查看浏览器中正在运行的应用程序,您将使用kubectl内置功能将本地请求转发到远程集群:

  • kubectl -n emojivoto port-forward svc/web-svc 8080:80

    kubectl -n emojivoto端口转发svc / web-svc 8080:80

Note: If you are not running this from your local machine, you will need to add the --address 0.0.0.0 flag to listen on all addresses and not only localhost.

注意:如果不是从本地计算机运行此命令,则需要添加--address 0.0.0.0标志以侦听所有地址,而不仅是localhost

Here you are again using kubectl in the emojivoto namespaces, but now calling the port-forward sub-command and directing it to forward all local requests on port 8080 to the Kubernetes service web-svc, on port 80. This is just a convenient way for you to access your application without needing to have a proper load balancer in place.

在这里,您再次在emojivoto命名空间中使用kubectl ,但是现在调用port-forward子命令并将其引导以将端口8080上的所有本地请求转发到端口80上的Kubernetes服务web-svc 。 这只是您访问应用程序的便捷方式,而无需安装适当的负载平衡器。

Now visit http://localhost:8080 and you will see the emojivoto application.

现在访问http://localhost:8080 ,您将看到emojivoto应用程序。

Press CTRL + C in your terminal. With an application running in your cluster, you are now ready to install Linkerd and see how it works.

在终端中按CTRL + C 。 在集群中运行应用程序后,您现在就可以安装Linkerd并查看其工作方式。

第2步-安装Linkerd (Step 2 — Installing Linkerd)

Now that you have an application running, let’s install Linkerd. To install it in your Kubernetes cluster, you first need the Linkerd CLI. You will use this command line interface to interact with Linkerd from your local machine. After that, you can install Linkerd in your cluster.

现在您已在运行应用程序,让我们安装Linkerd。 要将其安装在Kubernetes集群中,首先需要Linkerd CLI。 您将使用此命令行界面从本地计算机与Linkerd进行交互。 之后,您可以在群集中安装Linkerd。

First, let’s install the CLI with the script provided by the Linkerd team:

首先,让我们使用Linkerd团队提供的脚本安装CLI:

  • curl https://run.linkerd.io/install | sh

    curl https://run.linkerd.io/install | SH

Here you are using curl to download the installation script, and then you are piping the output to sh, which automatically executes the script. Alternatively, you can download the CLI directly from Linkerd’s release page.

在这里,您使用curl来下载安装脚本,然后将输出传递到sh ,后者将自动执行该脚本。 或者,您可以直接从Linkerd的发行版页面下载CLI。

If you use the script then it will install Linkerd at ~/.linkerd2/bin. Now confirm that the CLI is working correctly:

如果使用脚本,它将在~/.linkerd2/bin安装Linkerd。 现在,确认CLI正常运行:

  • ~/.linkerd2/bin/linkerd version

    〜/ .linkerd2 / bin / linkerd版本

The command will output something like this:

该命令将输出如下内容:

  1. Output
  2. Client version: stable-2.7.1
  3. Server version: unavailable

Then, to make it easier to run the CLI, add this directory to your $PATH:

然后,为了更轻松地运行CLI,请将此目录添加到$PATH

  • export PATH=$PATH:$HOME/.linkerd2/bin

    导出PATH = $ PATH:$ HOME / .linkerd2 / bin

Now you can more directly run the commands, like the previous one:

现在,您可以像上一个命令一样直接运行命令:

  • linkerd version

    链接器版本

Finally, let’s install Linkerd in your Kubernetes cluster. The linkerd install command is used to generate all the necessary yaml manifests needed to run Linkerd, but it will not apply these manifests to your cluster. Run this command to inspect its output:

最后,让我们在您的Kubernetes集群中安装Linkerd。 linkerd install命令用于生成运行Linkerd所需的所有必要yaml清单,但不会将这些清单应用于您的集群。 运行以下命令以检查其输出:

  • linkerd install

    链接器安装

You will see a long output, listing all the yaml manifests for resources Linkerd needs to run. To apply these manifests to your cluster, run:

您会看到很长的输出,列出了Linkerd需要运行的资源的所有yaml清单。 要将这些清单应用于您的集群,请运行:

  • linkerd install | kubectl apply -f -

    linkerd安装| kubectl适用-f-

Running linkerd install will output all the manifests you saw previously. | then pipes this output directly to kubectl apply, which will apply them.

运行linkerd install将输出您之前看到的所有清单。 | 然后将此输出直接传递到kubectl apply ,它将应用它们。

After you run this command, kubectl apply will output a list of all the resources that were created.

运行此命令后, kubectl apply将输出所有已创建资源的列表。

To confirm that everything is running in your cluster, run linkerd check:

要确认集群中的所有程序都在运行,请运行linkerd check

  • linkerd check

    链接检查

This will run several checks against your cluster to confirm that all the necessary components are running:

这将对您的集群运行几项检查,以确认所有必需的组件都在运行:

  1. Output
  2. kubernetes-api
  3. --------------
  4. √ can initialize the client
  5. √ can query the Kubernetes API
  6. [...]
  7. control-plane-version
  8. ---------------------
  9. control plane is up-to-date
  10. control plane and cli versions match
  11. Status check results are

Finally, run this command to open the built-in Linkerd dashboard in your browser (remember you will need to provide the --address 0.0.0.0 flag if you are not running this from your local machine):

最后,运行以下命令以在浏览器中打开内置的Linkerd仪表板(请--address 0.0.0.0如果您不是从本地计算机运行此命令,则需要提供--address 0.0.0.0标志):

  • linkerd dashboard

    链接仪表板

Most of the information you see in the dashboard you can obtain using the Linkerd CLI. For example, run this command to see high-level stats deployments:

您可以使用Linkerd CLI在仪表板上看到的大多数信息都可以获取。 例如,运行以下命令以查看高级统计信息部署:

  • linkerd stat deployments -n linkerd

    链接器状态部署-n链接器

Here you are saying that you want the stats for deployments running in the linkerd namespace. These are Linkerd’s own components, and, interestingly, you can use Linkerd itself to monitor them. You can see stats like requests per second (RPS), success rate, latency, and more. You can also see a Meshed column, which indicates how many pods Linkerd has injected:

在这里,您要说的是希望linkerd名称空间中运行的部署的统计信息。 这些是Linkerd自己的组件,有趣的是,您可以使用Linkerd本身来监视它们。 您可以看到诸如每秒请求数(RPS),成功率,等待时间等统计信息。 您还可以看到一个Meshed列,这表明有多少pods Linkerd注入了:

  1. Output
  2. NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
  3. linkerd-controller 1/1 100.00% 0.4rps 1ms 87ms 98ms 5
  4. linkerd-destination 1/1 100.00% 0.3rps 1ms 2ms 2ms 13
  5. linkerd-grafana 1/1 100.00% 0.3rps 2ms 3ms 3ms 2
  6. linkerd-identity 1/1 100.00% 0.3rps 1ms 2ms 2ms 10
  7. linkerd-prometheus 1/1 100.00% 0.7rps 35ms 155ms 191ms 9
  8. linkerd-proxy-injector 1/1 100.00% 0.3rps 2ms 3ms 3ms 2
  9. linkerd-sp-validator 1/1 100.00% 0.3rps 1ms 5ms 5ms 2
  10. linkerd-tap 1/1 100.00% 0.3rps 1ms 4ms 4ms 6
  11. linkerd-web 1/1 100.00% 0.3rps 1ms 2ms 2ms 2

Now try this command in your emojivoto namespace:

现在在您的emojivoto名称空间中尝试以下命令:

  • linkerd stat deployments -n emojivoto

    链接器统计部署-n emojivoto

Even though you can see your four services, none of the stats you saw before are available for these deployments, and in the “Meshed” column you can see that it says 0/1:

即使您可以看到四个服务,但之前的统计信息都不适用于这些部署,并且在“网格”列中,您可以看到它显示为0/1

  1. Output
  2. NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
  3. emoji 0/1 - - - - - -
  4. vote-bot 0/1 - - - - - -
  5. voting 0/1 - - - - - -
  6. web 0/1 - - - - - -

The output here means that you still have not injected Linkerd into the application. This will be your next step.

这里的输出意味着您仍然没有将Linkerd注入到应用程序中。 这是您的下一步。

步骤3 —将Linkerd注入您的应用程序 (Step 3 — Injecting Linkerd into Your Application)

Now that you have Linkerd running in your cluster, you are ready to inject it into your emojivoto application.

现在您已经在群集中运行了Linkerd,现在可以将其注入到emojivoto应用程序中了。

Linkerd works by running a sidecar container in your Kubernetes pods. That is, you will inject a linkerd proxy container into every pod you have running. Every request that your pods then send or receive will go through this very lightweight proxy that can gather metrics (like success rate, requests per second, and latency) and enforce policies (like timeouts and retries).

Linkerd的工作原理是在你的Kubernetes运行的三轮容器pods 。 也就是说,您会将链接器代理容器注入正在运行的每个pod 。 您的pods随后发送或接收的每个请求都将通过这个非常轻量级的代理,该代理可以收集指标(例如成功率,每秒请求和延迟)并强制执行策略(例如超时和重试)。

You can manually inject Linkerd’s proxy with this command:

您可以使用以下命令手动注入Linkerd的代理:

  • kubectl get deployments -n emojivoto -o yaml | linkerd inject - | kubectl apply -f -

    kubectl获取部署-n emojivoto -o yaml | 链接器注入-| kubectl适用-f-

In this command, you are first using kubectl get to get all the Kubernetes deployments you have running in the emojivoto namespace, and then specifying that you want the output in the yaml format. Then you are sending that output to the linkerd inject command. This command will read the yaml file with the current manifests you have running and modify it to include the linkerd proxy alongside every deployment.

在此命令中,您首先要使用kubectl get获取在emojivoto名称空间中运行的所有Kubernetes deployment ,然后指定要使用yaml格式的输出。 然后,您将该输出发送到linkerd inject命令。 该命令将读取具有您正在运行的当前清单的yaml文件,并对其进行修改以在每个deployment旁边包括linkerd代理。

Lastly, you are receiving this modified manifest and applying it to your cluster with kubectl apply.

最后,您将收到此修改的清单,并使用kubectl apply将其应用于您的集群。

After running this command you will see a message saying that all four emojivoto services (emoji, vote-bot, voting, and web) were successfully injected.

运行此命令后,您将看到一条消息,说明所有四个emojivoto服务( emojivote-botvotingweb )均已成功注入。

If you now retrieve stats for emojivoto, you will see that all your deployments are now meshed, and after a few seconds you will start to see the same stats that you saw for the linkerd namespace:

如果现在检索emojivoto stats ,您将看到所有deployment都已网格化,几秒钟后,您将开始看到与linkerd命名空间相同的统计信息:

  • linkerd stat deployments -n emojivoto

    链接器统计部署-n emojivoto

Here you can see the stats for all the four services that compose the emojivoto application, with their respective success rate, requests per second and latency, without having to write or change any application code.

在这里,您可以查看组成emojivoto应用程序的所有四个服务的统计信息,以及它们各自的成功率,每秒请求数和延迟,而无需编写或更改任何应用程序代码。

  1. Output
  2. NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN
  3. emoji 1/1 100.00% 1.9rps 1ms 2ms 2ms 2
  4. vote-bot 1/1 - - - - - -
  5. voting 1/1 85.96% 0.9rps 1ms 1ms 1ms 2
  6. web 1/1 93.04% 1.9rps 8ms 27ms 29ms 2

The vote-bot service doesn’t show any stats because it is just a bot that sends requests to other services and therefore is not receiving any traffic, which is in itself valuable information.

vote-bot服务没有显示任何统计信息,因为它只是一个向其他服务发送请求的机器人,因此没有收到任何流量,而流量本身就是有价值的信息。

Now let’s see how you can provide some extra information to Linkerd about your services to customize their behavior.

现在,让我们看看如何向Linkerd提供有关您的服务的一些额外信息,以自定义其行为。

步骤4 —定义服务配置文件 (Step 4 — Defining a Service Profile)

Now that you have injected Linkerd into your application, you can begin to retrieve valuable information about how each of your services is behaving. Moreover, you have accomplished this without needing to write any custom configurations or change your application’s code. However, if you do provide Linkerd with some additional information, it can then enforce numerous policies, like timeouts and retries. It can also then provide per-route metrics.

现在,您已经将Linkerd注入到您的应用程序中,您可以开始检索有关每个服务行为方式的有价值的信息。 而且,您无需编写任何自定义配置或更改应用程序的代码即可完成此操作。 但是,如果确实为Linkerd提供了一些其他信息,则它可以强制执行大量策略,例如超时和重试。 然后,它还可以提供每个路由的指标。

This information is provided through a Service Profile, which is a custom Linkerd resource where you can describe the routes in your applications and how each one will behave.

此信息通过服务配置文件提供,服务配置文件是自定义的Linkerd资源,您可以在其中描述应用程序中的路由以及每个路由的行为方式。

Here’s an example of what a Service Profile manifest looks like:

这是服务档案清单清单的示例:

example-service-profile.yaml
example-service-profile.yaml
  1. apiVersion: linkerd.io/v1alpha2
  2. kind: ServiceProfile
  3. metadata:
  4. name: my-service.my-namespace.svc.cluster.local
  5. spec:
  6. routes:
  7. - name: My Route Name
  8. isRetryable: true # Define it's safe to retry this route
  9. timeout: 100ms # Define a timeout for this route
  10. condition:
  11. method: GET
  12. pathRegex: /my/route/path

The Service Profile describes a list of routes, and then defines how requests that match the specified condition will behave. In this example, you are saying that every GET request sent to /my/route/path will timeout after 100ms, and, if they fail, they can be retried.

服务配置文件描述了路由列表,然后定义了与指定condition匹配的请求的行为。 在此示例中,您说的是发送到/my/route/path每个GET请求都会在100毫秒后超时,如果失败,则可以重试。

Let’s now create a Service Profile for one of your services. Taking voting-svc as an example, first use the Linkerd CLI to check the routes you have defined for this service:

现在让我们为您的一项服务创建服务档案。 以voting-svc为例,首先使用Linkerd CLI检查为此服务定义的路由:

  • linkerd routes svc/voting-svc -n emojivoto

    链接器路由svc / voting-svc -n emojivoto

Here you are using the linkerd routes command to list all the routes for the service voting-svc, in the emojiovoto namespace:

在这里,您使用linkerd routes命令列出emojiovoto命名空间中服务voting-svc所有路由:

  1. Output
  2. ROUTE SERVICE SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99
  3. [DEFAULT] voting-svc 83.05% 1.0rps 1ms 1ms 2ms

You will find only one route, [DEFAULT]. This is where all requests are grouped until you define your Service Profile.

您只会找到一条路线[DEFAULT] 。 在此定义所有请求,直到您定义服务配置文件。

Now open nano or your favorite editor to create a service-profile.yaml file:

现在打开nano或您喜欢的编辑器来创建service-profile.yaml文件:

  • nano service-profile.yaml

    纳米服务配置文件

Add the following Service Profile definition into this file:

将以下服务配置文件定义添加到此文件中:

service-profile.yaml
service-profile.yaml
  1. apiVersion: linkerd.io/v1alpha2
  2. kind: ServiceProfile
  3. metadata:
  4. name: voting-svc.emojivoto.svc.cluster.local
  5. namespace: emojivoto
  6. spec:
  7. routes:
  8. - name: VoteDoughnut
  9. isRetryable: true
  10. timeout: 100ms
  11. condition:
  12. method: POST
  13. pathRegex: /emojivoto.v1.VotingService/VoteDoughnut

Now save the file and close your editor.

现在保存文件并关闭编辑器。

Here you are declaring a Service Profile for the voting-svc service, in the emojivoto namespace. You have defined one route, called VoteDoughnut, which will match any POST request to the /emojivoto.v1.VotingService/VoteDoughnut path. If a request matching these criteria takes more than 100ms, Linkerd will cancel it and the client will receive a 504 response back. You are also telling Linkerd that if this request fails it can be retried.

在这里,您要在emojivoto名称空间中声明voting-svc服务的服务配置文件。 您已经定义了一个名为VoteDoughnut路由,该路由会将任何POST请求与/emojivoto.v1.VotingService/VoteDoughnut路径相匹配。 如果符合这些条件的请求花费了100毫秒以上,Linkerd将取消该请求,并且客户端将收到504响应。 您还告诉Linkerd,如果此请求失败,则可以重试。

Now apply this file to your cluster:

现在将此文件应用于您的集群:

  • kubectl apply -f service-profile.yaml

    kubectl套用-f service-profile.yaml

After a few seconds, recheck the routes for this service:

几秒钟后,重新检查此服务的路由:

  • linkerd routes svc/voting-svc -n emojivoto

    链接器路由svc / voting-svc -n emojivoto

You will now see your newly defined VoteDoughnut route:

现在,您将看到新定义的VoteDoughnut路线:

  1. Output
  2. ROUTE SERVICE SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99
  3. VoteDoughnut voting-svc 0.00% 0.2rps 1ms 1ms 1ms
  4. [DEFAULT] voting-svc 100.00% 0.8rps 1ms 4ms 4ms

You can see several custom metrics, like success rate, requests per second, and latency for this specific route. Note that the VoteDoughnut endpoint is intentionally configured to always return an error, and that it is outputting a success rate of 0%, while the [DEFAULT] route is outputting 100%.

您可以看到几个自定义指标,例如成功率,每秒请求数和此特定路由的延迟。 请注意, VoteDoughnut端点被有意配置为始终返回错误,并且其输出的成功率为0%,而[DEFAULT]路由的输出为100%。

So now, after giving Linkerd a little bit of information about your service, you have custom metrics per route, as well as two policies enforced: timeouts and retries.

因此,现在,在向Linkerd提供有关您的服务的一些信息之后,您就可以对每条路由自定义指标,并执行两个策略:超时和重试。

结论 (Conclusion)

In this article, you installed Linkerd in your Kubernetes cluster and used it to monitor a sample application. You extracted useful telemetry information like success rate, throughput, and latency. You also configured a Linkerd Service Profile to collect per-route metrics and enforce two policies in the emojivoto application.

在本文中,您将Linkerd安装在Kubernetes集群中,并用它来监视示例应用程序。 您提取了有用的遥测信息,例如成功率,吞吐量和延迟。 您还配置了Linkerd服务配置文件,以收集每个路由指标并在emojivoto应用程序中实施两个策略。

If you are interested in learning more about Linkerd, you can browse their great documentation page, where they show how to secure your services, configure distributed tracing, automate canary releases, and much more.

如果您想了解有关Linkerd的更多信息,可以浏览其出色的文档页面 ,他们在该页面上显示如何保护您的服务配置分布式跟踪自动执行Canary发布等等。

From here you might also consider checking out Istio, which is another Service Mesh with a different set of features and trade-offs.

从这里,您还可以考虑签出Istio ,这是另一个具有不同功能和折衷方案的Service Mesh。

翻译自: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-linkerd-with-kubernetes

安装kubernetes

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/小桥流水78/article/detail/915796
推荐阅读
相关标签
  

闽ICP备14008679号