Python—自动化部署工具：Fabric

Fabric是python编写的一款自动化部署工具

Fabric依赖paramiko进行SSH交互，某种意义上Fabric是对paramiko的封装，封装完成后，不需要像使用paramiko一样处理SSH连接，只需专心于自己的需求即可。

Fabric的设计思路的是提供简单的API来完成所有部署，因此，Fabric对基本的系统管理操作进行了封装。

本篇文章主要针对Python3

fabric最常用的用法是通过SSH连接远程服务器执行Shell命令，然后拿到结果（可选），默认情况下，远程命令的输出直接被捕获并打印在终端上，以下为官网示例：

>>> from fabric import Connection
>>> c = Connection('web1')
>>> result = c.run('uname -s')
Linux
>>> result.stdout.strip() == 'Linux'
True
>>> result.exited
0
>>> result.ok
True
>>> result.command
'uname -s'
>>> result.connection
<Connection host=web1>
>>> result.connection.host
'web1'

这里遇到问题，安装上述方式并不能成功直接报错：

In [46]: c = Connection('host2')

In [48]: result = c.run("uname -s")
---------------------------------------------------------------------------
SSHException                              Traceback (most recent call last)
<ipython-input-48-688064d71ccd> in <module>
----> 1 result = c.run("uname -s")

<decorator-gen-3> in run(self, command, **kwargs)

/usr/python/lib/python3.7/site-packages/fabric/connection.py in opens(method, self, *args, **kwargs)
     27 @decorator
     28 def opens(method, self, *args, **kwargs):
---> 29     self.open()
     30     return method(self, *args, **kwargs)
     31

/usr/python/lib/python3.7/site-packages/fabric/connection.py in open(self)
    613             del kwargs["key_filename"]
    614         # Actually connect!
--> 615         self.client.connect(**kwargs)
    616         self.transport = self.client.get_transport()
    617

/usr/python/lib/python3.7/site-packages/paramiko/client.py in connect(self, hostname, port, username, password, pkey, key_filename, timeout, allow_agent, look_for_keys, compress, sock, gss_auth, gss_kex, gss_deleg_creds, gss_host, banner_timeout, auth_timeout, gss_trust_dns, passphrase)
    435             gss_deleg_creds,
    436             t.gss_host,
--> 437             passphrase,
    438         )
    439

/usr/python/lib/python3.7/site-packages/paramiko/client.py in _auth(self, username, password, pkey, key_filenames, allow_agent, look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host, passphrase)
    748         if saved_exception is not None:
    749             raise saved_exception
--> 750         raise SSHException("No authentication methods available")
    751
    752     def _log(self, level, msg):

查看fabric connection源码：

connection.py源码如下：

class Connection(Context):
    """
    A connection to an SSH daemon, with methods for commands and file transfer.

    **Basics**

    This class inherits from Invoke's `~invoke.context.Context`, as it is a
    context within which commands, tasks etc can operate. It also encapsulates
    a Paramiko `~paramiko.client.SSHClient` instance, performing useful high
    level operations with that `~paramiko.client.SSHClient` and
    `~paramiko.channel.Channel` instances generated from it.

    **Lifecycle**

    `.Connection` has a basic "`create <__init__>`, `connect/open <open>`, `do
    work <run>`, `disconnect/close <close>`" lifecycle:

    - `Instantiation <__init__>` imprints the object with its connection
      parameters (but does **not** actually initiate the network connection).

        - An alternate constructor exists for users :ref:`upgrading piecemeal
          from Fabric 1 <from-v1>`: `from_v1`

    - Methods like `run`, `get` etc automatically trigger a call to
      `open` if the connection is not active; users may of course call `open`
      manually if desired.
    - Connections do not always need to be explicitly closed; much of the
      time, Paramiko's garbage collection hooks or Python's own shutdown
      sequence will take care of things. **However**, should you encounter edge
      cases (for example, sessions hanging on exit) it's helpful to explicitly
      close connections when you're done with them.

      This can be accomplished by manually calling `close`, or by using the
      object as a contextmanager::

        with Connection('host') as c:
            c.run('command')
            c.put('file')

    .. note::
        This class rebinds `invoke.context.Context.run` to `.local` so both
        remote and local command execution can coexist.

    **Configuration**

    Most `.Connection` parameters honor :doc:`Invoke-style configuration
    </concepts/configuration>` as well as any applicable :ref:`SSH config file
    directives <connection-ssh-config>`. For example, to end up with a
    connection to ``admin@myhost``, one could:

    - Use any built-in config mechanism, such as ``/etc/fabric.yml``,
      ``~/.fabric.json``, collection-driven configuration, env vars, etc,
      stating ``user: admin`` (or ``{"user": "admin"}``, depending on config
      format.) Then ``Connection('myhost')`` would implicitly have a ``user``
      of ``admin``.
    - Use an SSH config file containing ``User admin`` within any applicable
      ``Host`` header (``Host myhost``, ``Host *``, etc.) Again,
      ``Connection('myhost')`` will default to an ``admin`` user.
    - Leverage host-parameter shorthand (described in `.Config.__init__`), i.e.
      ``Connection('admin@myhost')``.
    - Give the parameter directly: ``Connection('myhost', user='admin')``.

    The same applies to agent forwarding, gateways, and so forth.

    .. versionadded:: 2.0
    """

    # NOTE: these are initialized here to hint to invoke.Config.__setattr__
    # that they should be treated as real attributes instead of config proxies.
    # (Additionally, we're doing this instead of using invoke.Config._set() so
    # we can take advantage of Sphinx's attribute-doc-comment static analysis.)
    # Once an instance is created, these values will usually be non-None
    # because they default to the default config values.
    host = None
    original_host = None
    user = None
    port = None
    ssh_config = None
    gateway = None
    forward_agent = None
    connect_timeout = None
    connect_kwargs = None
    client = None
    transport = None
    _sftp = None
    _agent_handler = None

    @classmethod
    def from_v1(cls, env, **kwargs):
        """
        Alternate constructor which uses Fabric 1's ``env`` dict for settings.

        All keyword arguments besides ``env`` are passed unmolested into the
        primary constructor.

        .. warning::
            Because your own config overrides will win over data from ``env``,
            make sure you only set values you *intend* to change from your v1
            environment!

        For details on exactly which ``env`` vars are imported and what they
        become in the new API, please see :ref:`v1-env-var-imports`.

        :param env:
            An explicit Fabric 1 ``env`` dict (technically, any
            ``fabric.utils._AttributeDict`` instance should work) to pull
            configuration from.

        .. versionadded:: 2.4
        """
        # TODO: import fabric.state.env (need good way to test it first...)
        # TODO: how to handle somebody accidentally calling this in a process
        # where 'fabric' is fabric 2, and there's no fabric 1? Probably just a
        # re-raise of ImportError??
        # Our only requirement is a non-empty host_string
        if not env.host_string:
            raise InvalidV1Env(
                "Supplied v1 env has an empty `host_string` value! Please make sure you're calling Connection.from_v1 within a connected Fabric 1 session."  # noqa
            )
        # TODO: detect collisions with kwargs & except instead of overwriting?
        # (More Zen of Python compliant, but also, effort, and also, makes it
        # harder for users to intentionally overwrite!)
        connect_kwargs = kwargs.setdefault("connect_kwargs", {})
        kwargs.setdefault("host", env.host_string)
        shorthand = derive_shorthand(env.host_string)
        # TODO: don't we need to do the below skipping for user too?
        kwargs.setdefault("user", env.user)
        # Skip port if host string seemed to have it; otherwise we hit our own
        # ambiguity clause in __init__. v1 would also have been doing this
        # anyways (host string wins over other settings).
        if not shorthand["port"]:
            # Run port through int(); v1 inexplicably has a string default...
            kwargs.setdefault("port", int(env.port))
        # key_filename defaults to None in v1, but in v2, we expect it to be
        # either unset, or set to a list. Thus, we only pull it over if it is
        # not None.
        if env.key_filename is not None:
            connect_kwargs.setdefault("key_filename", env.key_filename)
        # Obtain config values, if not given, from its own from_v1
        # NOTE: not using setdefault as we truly only want to call
        # Config.from_v1 when necessary.
        if "config" not in kwargs:
            kwargs["config"] = Config.from_v1(env)
        return cls(**kwargs)

    # TODO: should "reopening" an existing Connection object that has been
    # closed, be allowed? (See e.g. how v1 detects closed/semi-closed
    # connections & nukes them before creating a new client to the same host.)
    # TODO: push some of this into paramiko.client.Client? e.g. expand what
    # Client.exec_command does, it already allows configuring a subset of what
    # we do / will eventually do / did in 1.x. It's silly to have to do
    # .get_transport().open_session().
    def __init__(
        self,
        host,
        user=None,
        port=None,
        config=None,
        gateway=None,
        forward_agent=None,
        connect_timeout=None,
        connect_kwargs=None,
        inline_ssh_env=None,
    ):
        """
        Set up a new object representing a server connection.

        :param str host:
            the hostname (or IP address) of this connection.

            May include shorthand for the ``user`` and/or ``port`` parameters,
            of the form ``user@host``, ``host:port``, or ``user@host:port``.

            .. note::
                Due to ambiguity, IPv6 host addresses are incompatible with the
                ``host:port`` shorthand (though ``user@host`` will still work
                OK). In other words, the presence of >1 ``:`` character will
                prevent any attempt to derive a shorthand port number; use the
                explicit ``port`` parameter instead.

            .. note::
                If ``host`` matches a ``Host`` clause in loaded SSH config
                data, and that ``Host`` clause contains a ``Hostname``
                directive, the resulting `.Connection` object will behave as if
                ``host`` is equal to that ``Hostname`` value.

                In all cases, the original value of ``host`` is preserved as
                the ``original_host`` attribute.

                Thus, given SSH config like so::

                    Host myalias
                        Hostname realhostname

                a call like ``Connection(host='myalias')`` will result in an
                object whose ``host`` attribute is ``realhostname``, and whose
                ``original_host`` attribute is ``myalias``.

        :param str user:
            the login user for the remote connection. Defaults to
            ``config.user``.

        :param int port:
            the remote port. Defaults to ``config.port``.

        :param config:
            configuration settings to use when executing methods on this
            `.Connection` (e.g. default SSH port and so forth).

            Should be a `.Config` or an `invoke.config.Config`
            (which will be turned into a `.Config`).

            Default is an anonymous `.Config` object.

        :param gateway:
            An object to use as a proxy or gateway for this connection.

            This parameter accepts one of the following:

            - another `.Connection` (for a ``ProxyJump`` style gateway);
            - a shell command string (for a ``ProxyCommand`` style style
              gateway).

            Default: ``None``, meaning no gatewaying will occur (unless
            otherwise configured; if one wants to override a configured gateway
            at runtime, specify ``gateway=False``.)

            .. seealso:: :ref:`ssh-gateways`

        :param bool forward_agent:
            Whether to enable SSH agent forwarding.

            Default: ``config.forward_agent``.

        :param int connect_timeout:
            Connection timeout, in seconds.

            Default: ``config.timeouts.connect``.

        :param dict connect_kwargs:
            Keyword arguments handed verbatim to
            `SSHClient.connect <paramiko.client.SSHClient.connect>` (when
            `.open` is called).

            `.Connection` tries not to grow additional settings/kwargs of its
            own unless it is adding value of some kind; thus,
            ``connect_kwargs`` is currently the right place to hand in
            parameters such as ``pkey`` or ``key_filename``.

            Default: ``config.connect_kwargs``.

        :param bool inline_ssh_env:
            Whether to send environment variables "inline" as prefixes in front
            of command strings (``export VARNAME=value && mycommand here``),
            instead of trying to submit them through the SSH protocol itself
            (which is the default behavior). This is necessary if the remote
            server has a restricted ``AcceptEnv`` setting (which is the common
            default).

            The default value is the value of the ``inline_ssh_env``
            :ref:`configuration value <default-values>` (which itself defaults
            to ``False``).

            .. warning::
                This functionality does **not** currently perform any shell
                escaping on your behalf! Be careful when using nontrivial
                values, and note that you can put in your own quoting,
                backslashing etc if desired.

                Consider using a different approach (such as actual
                remote shell scripts) if you run into too many issues here.

            .. note::
                When serializing into prefixed ``FOO=bar`` format, we apply the
                builtin `sorted` function to the env dictionary's keys, to
                remove what would otherwise be ambiguous/arbitrary ordering.

            .. note::
                This setting has no bearing on *local* shell commands; it only
                affects remote commands, and thus, methods like `.run` and
                `.sudo`.

        :raises ValueError:
            if user or port values are given via both ``host`` shorthand *and*
            their own arguments. (We `refuse the temptation to guess`_).

        .. _refuse the temptation to guess:
            http://zen-of-python.info/
            in-the-face-of-ambiguity-refuse-the-temptation-to-guess.html#12

        .. versionchanged:: 2.3
            Added the ``inline_ssh_env`` parameter.
        """
        # NOTE: parent __init__ sets self._config; for now we simply overwrite
        # that below. If it's somehow problematic we would want to break parent
        # __init__ up in a manner that is more cleanly overrideable.
        super(Connection, self).__init__(config=config)

        #: The .Config object referenced when handling default values (for e.g.
        #: user or port, when not explicitly given) or deciding how to behave.
        if config is None:
            config = Config()
        # Handle 'vanilla' Invoke config objects, which need cloning 'into' one
        # of our own Configs (which grants the new defaults, etc, while not
        # squashing them if the Invoke-level config already accounted for them)
        elif not isinstance(config, Config):
            config = config.clone(into=Config)
        self._set(_config=config)
        # TODO: when/how to run load_files, merge, load_shell_env, etc?
        # TODO: i.e. what is the lib use case here (and honestly in invoke too)

        shorthand = self.derive_shorthand(host)
        host = shorthand["host"]
        err = "You supplied the {} via both shorthand and kwarg! Please pick one."  # noqa
        if shorthand["user"] is not None:
            if user is not None:
                raise ValueError(err.format("user"))
            user = shorthand["user"]
        if shorthand["port"] is not None:
            if port is not None:
                raise ValueError(err.format("port"))
            port = shorthand["port"]

        # NOTE: we load SSH config data as early as possible as it has
        # potential to affect nearly every other attribute.
        #: The per-host SSH config data, if any. (See :ref:`ssh-config`.)
        self.ssh_config = self.config.base_ssh_config.lookup(host)

        self.original_host = host
        #: The hostname of the target server.
        self.host = host
        if "hostname" in self.ssh_config:
            # TODO: log that this occurred?
            self.host = self.ssh_config["hostname"]

        #: The username this connection will use to connect to the remote end.
        self.user = user or self.ssh_config.get("user", self.config.user)
        # TODO: is it _ever_ possible to give an empty user value (e.g.
        # user='')? E.g. do some SSH server specs allow for that?

        #: The network port to connect on.
        self.port = port or int(self.ssh_config.get("port", self.config.port))

        # Gateway/proxy/bastion/jump setting: non-None values - string,
        # Connection, even eg False - get set directly; None triggers seek in
        # config/ssh_config
        #: The gateway `.Connection` or ``ProxyCommand`` string to be used,
        #: if any.
        self.gateway = gateway if gateway is not None else self.get_gateway()
        # NOTE: we use string above, vs ProxyCommand obj, to avoid spinning up
        # the ProxyCommand subprocess at init time, vs open() time.
        # TODO: make paramiko.proxy.ProxyCommand lazy instead?

        if forward_agent is None:
            # Default to config...
            forward_agent = self.config.forward_agent
            # But if ssh_config is present, it wins
            if "forwardagent" in self.ssh_config:
                # TODO: SSHConfig really, seriously needs some love here, god
                map_ = {"yes": True, "no": False}
                forward_agent = map_[self.ssh_config["forwardagent"]]
        #: Whether agent forwarding is enabled.
        self.forward_agent = forward_agent

        if connect_timeout is None:
            connect_timeout = self.ssh_config.get(
                "connecttimeout", self.config.timeouts.connect
            )
        if connect_timeout is not None:
            connect_timeout = int(connect_timeout)
        #: Connection timeout
        self.connect_timeout = connect_timeout

        #: Keyword arguments given to `paramiko.client.SSHClient.connect` when
        #: `open` is called.
        self.connect_kwargs = self.resolve_connect_kwargs(connect_kwargs)

        #: The `paramiko.client.SSHClient` instance this connection wraps.
        client = SSHClient()
        client.set_missing_host_key_policy(AutoAddPolicy())
        self.client = client

        #: A convenience handle onto the return value of
        #: ``self.client.get_transport()``.
        self.transport = None

	if inline_ssh_env is None:
            inline_ssh_env = self.config.inline_ssh_env
        #: Whether to construct remote command lines with env vars prefixed
        #: inline.
        self.inline_ssh_env = inline_ssh_env

connection 成员变量：

host = None             # 主机名或IP地址: www.host.com, 66.66.66.66
original_host = None    # 同host
user = None             # 系统用户名: root, someone
port = None             # 端口号（远程执行某些应用需提供）
gateway = None          # 网关
forward_agent = None    # 代理
connect_timeout = None  # 超时时间
connect_kwargs = None   # 连接参数 重要

client = None           # 客户端

构造函数参数：

Connection.__init__()

host
user=None
port=None
config=None
gateway=None
forward_agent=None
connect_timeout=None
connect_kwargs=None

这里比较重要的参数是 config和 connection_kwargs

构造函数主体：

config

super(Connection, self).__init__(config=config)
if config is None:
    config = Config()
elif not isinstance(config, Config):
    config = config.clone(into=Config)
self._set(_config=config)

config成员变量是一个 Config对象，它是调用父类 Context.__init__()方法来初始化的。 Context.__init__()定义如下:

class Context(DataProxy):
    def __init__(self, config=None):
        config = config if config is not None else Config()
        self._set(_config=config)

        command_prefixes = list()
        self._set(command_prefixes=command_prefixes)

        command_cwds = list()
        self._set(command_cwds=command_cwds)

具体过程是 Context.__init__()初始化时调用 _set()绑定了 Config成员对象 _config:

def _set(self, *args, **kwargs):
    if args:
        object.__setattr__(self, *args)
    for key, value in six.iteritems(kwargs):
        object.__setattr__(self, key, value)

再通过加了 @property的 config()函数，使得 connection对象能直接用 self.config来引用 _config:

@property
def config(self):
    return self._config

@config.setter
def config(self, value):
    self._set(_config=value)

host, user, port

shorthand = self.derive_shorthand(host)
host = shorthand["host"]
err = (
    "You supplied the {} via both shorthand and kwarg! Please pick one."  # noqa
)
if shorthand["user"] is not None:
    if user is not None:
        raise ValueError(err.format("user"))
    user = shorthand["user"]
if shorthand["port"] is not None:
    if port is not None:
        raise ValueError(err.format("port"))
    port = shorthand["port"]

这里是处理host参数的， host可以有一下几种传参形式

user@host:port  # 例如: root@10.10.10.10:6666
user@host       # 例如: root@10.10.10.10
host:port       # 例如: 10.10.10.10:6666
host            # 例如: 10.10.10.10

前三种会调用 self.derive_shorthand(host)分别解析出 self.host， self.user和 self.port，最后一种需单独传入 user，port。
如果用前三种传入方式的话，不能再重复传入 user或 port了，会抛出异常

以上分析

定位报错位置：

kwargs = dict(
    self.connect_kwargs,
    username=self.user,
    hostname=self.host,
    port=self.port,
)
if self.gateway:
    kwargs["sock"] = self.open_gateway()
if self.connect_timeout:
    kwargs["timeout"] = self.connect_timeout
# Strip out empty defaults for less noisy debugging
if "key_filename" in kwargs and not kwargs["key_filename"]:
    del kwargs["key_filename"]
# Actually connect!

/usr/python/lib/python3.7/site-packages/paramiko/client.py in connect(self, hostname, port, username, password, pkey, key_filename, timeout, allow_agent, look_for_keys, compress, sock, gss_auth, gss_kex, gss_deleg_creds, gss_host, banner_timeout, auth_timeout, gss_trust_dns, passphrase)
    435             gss_deleg_creds,
    436             t.gss_host,
--> 437             passphrase,
    438         )

可以看到，在执行connect方法的时候解析参数错误，这里我们没有传递passphrase参数，导致ssh连接报错

传参的时候是将kwargs传了过去，刚才我们的参数里面缺少self.connect_kwargs这个参数

connect的定义为：

def connect(
        self,
        hostname,
        port=SSH_PORT,
        username=None,
        password=None,      # 你
        pkey=None,          # 你
        key_filename=None,  # 还有你
        timeout=None,
        allow_agent=True,
        look_for_keys=True,
        compress=False,
        sock=None,
        gss_auth=False,
        gss_kex=False,
        gss_deleg_creds=True,
        gss_host=None,
        banner_timeout=None,
        auth_timeout=None,
        gss_trust_dns=True,
        passphrase=None,
    )

使用password方式:

In [27]: c = Connection('47.104.148.179',user='root', connect_kwargs={'password':'your password'})

In [28]: result = c.run('uname -s')
Linux

In [29]: result.stdout.strip() == "Linux"
Out[29]: True

In [30]: result.exited
Out[30]: 0

In [31]: result.ok
Out[31]: True

In [32]: result.command
Out[32]: 'uname -s'

In [33]: result.connection
Out[33]: <Connection host=47.104.148.179>

In [39]: result.connection.host
Out[39]: '47.104.148.179

使用key_filename方式：

In [11]: c = Connection('47.104.148.179', user='root', connect_kwargs={'key_filename':'/root/.ssh/authorized_keys'}
    ...: )

In [12]: c.run("uname -s")
Linux
Out[12]: <Result cmd='uname -s' exited=0>

In [13]: c.run("ls")
coding_time
comment_tree
python_document_manage
python_linux_automation
python_linux_manage
python_linux_monitor
python_linux_network_manage
sys_back
sys_manager
Out[13]: <Result cmd='ls' exited=0>

通过run命令使用sudo提权执行命令

>>> from fabric import Connection
>>> c = Connection('db1')
>>> c.run('sudo useradd mydbuser', pty=True)
[sudo] password:
<Result cmd='sudo useradd mydbuser' exited=0>
>>> c.run('id -u mydbuser')
1001
<Result cmd='id -u mydbuser' exited=0>

auto-response

自动响应：

当用户是普通用户的时候，可以使用run里面的watchers用法，进行自动响应

添加用户

In [21]: c.run('useradd mydbuser', pty=True)
Out[21]: <Result cmd='useradd mydbuser' exited=0>

In [23]: c.run('id mydbuser')
uid=1003(mydbuser) gid=1003(mydbuser) groups=1003(mydbuser)
Out[23]: <Result cmd='id mydbuser' exited=0>

执行命令

In [21]: from invoke import Responder

In [22]: from fabric import Connection

In [23]: c = Connection('47.104.148.179', user='ykyk', connect_kwargs={'password':'123456'})

   In [30]: sudopass = Responder(
        ...:     pattern=r'

$$sudo$$ password for ykyk:',
        ...:     response='xxxxxxx\n',
        ...:

   In [29]: c.run('sudo whoami', pty=True, watchers=[sudopass])

   [sudo] password for ykyk: root

   Out[29]: <Result cmd='sudo whoami' exited=0>

高级用法：

watchers/responders 在上一步很有效，但是每次使用使用时都要设置一次模板，在实际环境中不够便利，

Invoke提供 Context.sudo 方法，这个方法能够处理大部分常用情况，而不会越权

使用这个方法之前必须保证用户密码已经存储在环境变量中，剩余的就可以交给Connection.sudo来解决

示例如下：

>>> import getpass
>>> from fabric import Connection, Config
>>> sudo_pass = getpass.getpass("What's your sudo password?")
What's your sudo password?
>>> config = Config(overrides={'sudo': {'password': sudo_pass}})
>>> c = Connection('db1', config=config)
>>> c.sudo('whoami', hide='stderr')
root
<Result cmd="...whoami" exited=0>
>>> c.sudo('useradd mydbuser')
<Result cmd="...useradd mydbuser" exited=0>
>>> c.run('id -u mydbuser')
1001
<Result cmd='id -u mydbuser' exited=0>

传输文件

In [1]: ls
coding_time    python_document_manage/   python_linux_manage/   python_linux_network_manage/  sys_manager/
comment_tree/  python_linux_automation/  python_linux_monitor/  sys_back/

In [2]: from fabric import Connection

In [3]: result = Connection('own').put('coding_time', remote='/tmp/')

In [4]: print('Upload {0.local} to {0.remote}'.format(result))
Upload /root/coding_time to /tmp/coding_time

多任务整合

示例：

当我们需要上传某个文件到服务器并解压到特定目录时，可以这样写：

In [1]: ls
binlog2sql-master/                          paramiko-master.zip                     vim81/
cclang/                                     Pydiction-master/                       vim-8.1.tar.bz2
c_study/                                    Pydiction-master.zip                    vim-master/
master.zip                                  pyenv-master.zip                        vim-master.zip
mysql-8.0.13-linux-glibc2.12-x86_64.tar.xz  pyenv-virtualenv-master.zip             vim-snipmate/
paramiko-master/                            rabbitmq-server-3.6.6-1.el7.noarch.rpm

In [2]: from fabric import Connection

In [3]: c = Connection('own')

In [4]: c.put('mysql-8.0.13-linux-glibc2.12-x86_64.tar.xz','/tmp')
Out[4]: <fabric.transfer.Result at 0x7fedf9e36518>

In [6]: c.run('tar xf /tmp/mysql-8.0.13-linux-glibc2.12-x86_64.tar.xz -C /tmp')
Out[6]: <Result cmd='tar xf /tmp/mysql-8.0.13-linux-glibc2.12-x86_64.tar.xz -C /tmp' exited=0>

这里我们可以直接封装成一个方法：

In [7]: def upload_file(c):
   ...:     c.put('mysql-8.0.13-linux-glibc2.12-x86_64.tar.xz','/tmp')
   ...:     c.run('tar xf /tmp/mysql-8.0.13-linux-glibc2.12-x86_64.tar.xz -C /tmp')

在多个服务器上执行命令

In [3]: for host in ('own', 'redis','mysql_test'):
   ...:     result = Connection(host).run('uname -s')
   ...:     print("{}: {}".format(host, result.stdout.strip()))
   ...:
Linux
own: Linux
Linux
redis: Linux
Linux
mysql_test: Linux

还可以使用fabric中的SerialGroup方法：

In [4]: from fabric import SerialGroup as Group

In [5]: results = Group('own', 'redis', 'mysql_test').run('uname -s')
Linux
Linux
Linux

In [8]: for connection, result in results.items():
   ...:     print("{0.host}: {1.stdout}".format(connection, result))
   ...:
   ...:
47.104.148.xx: Linux

116.62.195.xx: Linux

47.99.123.xx: Linux

集成到一起：

from fabric import SerialGroup as Group

def upload_and_unpack(c):
    if c.run('test -f /opt/mydata/myfile', warn=True).failed:
        c.put('myfiles.tgz', '/opt/mydata')
        c.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz')

for connection in Group('web1', 'web2', 'web3'):
    upload_and_unpack(connection)

fabric 命令行工具

fabric提供了一个类似Shell终端的工具：fab

fab执行命令时，默认引用一个名称为fabfile.py的文件，这个文件包含一个到多个函数，使用fab命令可以调用这些函数， 函数在fabric中成为task

下面给出fabfile.py的样例文件：

from fabric import task

@task
def hostname(c):
    c.run('hostname')

@task
def ls(path='.'):
    c.run('ls {}'.format(path))

def tail(path='/etc/passwd', line=10):
    sudo('tail -n {0}, {1}'.format(line, path))

注意： 新版本的fab取消了api，所以相应的方法较之旧版本使用起来更加简洁，许多方法较之以前变化较大

[root@ykyk python_linux_automation]# fab3 --list
Available tasks:

  hostname
  ls

获取服务器信息需要在命令行指定：

[root@ykyk python_linux_automation]# fab3 -H mysql_test hostname
izbp1cmbkj49ynx81cezu3z

[root@ykyk python_linux_automation]# fab3 -H mysql_test,own,redis  hostname
izbp1cmbkj49ynx81cezu3z
ykyk
izbp1a43b9q4zlsifma7muz

fab命令行参数：

[root@ykyk python_linux_automation]# fab3 --help 
Usage: fab3 [--core-opts] task1 [--task1-opts] ... taskN [--taskN-opts]

Core options:

  --complete                         Print tab-completion candidates for given parse remainder.
  --hide=STRING                      Set default value of run()'s 'hide' kwarg.
  --no-dedupe                        Disable task deduplication.
  --print-completion-script=STRING   Print the tab-completion script for your preferred shell (bash|zsh|fish).
  --prompt-for-login-password        Request an upfront SSH-auth password prompt.
  --prompt-for-passphrase            Request an upfront SSH key passphrase prompt.
  --prompt-for-sudo-password         Prompt user at start of session for the sudo.password config value.
  --write-pyc                        Enable creation of .pyc files.
  -c STRING, --collection=STRING     Specify collection name to load.
  -d, --debug                        Enable debug output.
  -D INT, --list-depth=INT           When listing tasks, only show the first INT levels.
  -e, --echo                         Echo executed commands before running.
  -f STRING, --config=STRING         Runtime configuration file to use.
  -F STRING, --list-format=STRING    Change the display format used when listing tasks. Should be one of: flat
                                     (default), nested, json.
  -h [STRING], --help[=STRING]       Show core or per-task help and exit.
  -H STRING, --hosts=STRING          Comma-separated host name(s) to execute tasks against.
  -i, --identity                     Path to runtime SSH identity (key) file. May be given multiple times.
  -l [STRING], --list[=STRING]       List available tasks, optionally limited to a namespace.
  -p, --pty                          Use a pty when executing shell commands.
  -r STRING, --search-root=STRING    Change root directory used for finding task modules.
  -S STRING, --ssh-config=STRING     Path to runtime SSH config file.
  -V, --version                      Show version and exit.
  -w, --warn-only                    Warn, instead of failing, when shell commands fail.

• pty

pty用于设置伪终端，如果执行命令后需要一个常驻的服务进程，需要设置为pty=False,避免因fabric退出而导致程序退出

fabric装饰器

1. fabric中的task

• task是fabric需要在远程服务器执行的任务，
  • 默认情况下，fabfile中的所有可调用对象都是task，python中的函数是一个可调用对象
  • 继承fabric的task类，不推荐
  • 使用fabric装饰器，注意：如果fabfile中定义了多个task，只有其中一个使用了task，那么其他notask函数不是task

role新版本取消了。使用SerialGroupruns_once 只运行一次

以上就是fabric的一些方法

-------------------------------------------------------------------------------------------------------------

案例：使用fabric源码安装redis

from fabric import task
from fabric import connection
from invoke import Exit
from invocations.console import confirm

hosts = ['own']


@task#(hosts='own')
def test(c):
    with c.prefix('cd /root/python_linux_automation/redis-4.0.9'):
        result = c.run('make &&  make test', warn=True, pty=False)
        if result.failed and not confirm('Tests failed, continue anyway?'):
            raise SystemExit("Aborting at user requeset")
        else:
            print('All tests passed without errors')
            c.run('make clean', warn=True, pty=False, hide=True)
    with c.prefix("cd /root/python_linux_automation/"):
        c.run('tar -czf redis-4.0.9.tar.gz redis-4.0.9')

@task
def deploy(c):
    c.put('redis-4.0.9.tar.gz', '/tmp/redis-4.0.9.tar.gz')
    with c.cd('/tmp'):
        c.run('tar xzf redis-4.0.9.tar.gz')
        with c.cd('redis-4.0.9'):
            c.run('make')
            with c.cd('src'):
                c.run('make install')

@task
def clean_file(c):
    with c.cd('/tmp'):
        c.run('rm -rf redis-4.0.9.tar.gz')
        c.run('rm -rf redis-4.0.9')

@task
def clean_local_file(c):
    with c.prefix('cd /root/python_linux_automation/'):
        c.run('rm -rf redis-4.0.9.tar.gz')

@task
def install(c):
    for host in hosts:
        c = connection.Connection('own')
        test(c)
        deploy(c)
        clean_file(c)
        clean_local_file(c)