nova api执行过程（以nova list为例）
本文仅供学习参考：
- 使用devstack部署的openstack，需了解如何在screen中重新启动一个服务
- 请注意时间，因为openstack处在不断更新中，本文学习的是当前最新的openstack版本
- 在screen下用ipdb进行调试
- 对openstack整个架构有个大致了解
- 没有写明根目录的默认根目录是/opt/stack/nova（即openstack源码目录下的nova目录）
REQ: curl -g -i -X GET http://10.238.158.189/identity -H “Accept: application/json” -H “User-Agent: nova keystoneauth1/2.12.1 python-requests/2.11.1 CPython/2.7.6”
REQ: curl -g -i -X GET http://10.238.158.189:8774/v2.1 -H “User-Agent: python-novaclient” -H “Accept: application/json” -H “X-Auth-Token: {SHA1}1ed8ae56fa11986c2d3aef62a58d3355d9178d2c”
REQ: curl -g -i -X GET http://10.238.158.189:8774/v2.1/servers/detail -H “OpenStack-API-Version: compute 2.37” -H “User-Agent: python-novaclient” -H “Accept: application/json” -H “X-OpenStack-Nova-API-Version: 2.37” -H “X-Auth-Token: {SHA1}1ed8ae56fa11986c2d3aef62a58d3355d9178d2c”
第一步：从Keystone拿到一个授权的token
第二步：把返回的token填入api请求中，该步为验证版本v2.1之类的（不确定）
第三步：同样需要将token填入api请求，这一个步骤是真正的主要的请求即获得所有当前活跃状态的虚拟机
nova list命令转化为HTTP请求
由novaclient实现，不关注…
HTTP请求到WSGI Application
这里需要了解下WSGI和paste配置文件，WSGI推荐 –> https://www.fullstackpython.com/wsgi-servers.html
Nova API服务nova-api（在screen中对应的编号为6的window）在启动 时（没有发出HTTP请求时），会根据Nova配置文件（/etc/nova/nova.conf）的enable_apis选项内容创建一个或多个WSGI Server，用devstack部署默认的配置如下
enabled_apis = osapi_compute,metadata
Paste Deploy会在各个WSGI Server创建使参与进来，基于Paste配置文件（/etc/nova/api-paste.ini）去加载WSGI Application，加载WSGI Application由nova/service.py实现，加载后，WSGI Application就处在等待和监听状态。
class WSGIService(service.Service):
"""Provides ability to launch API from a 'paste' configuration."""
def __init__(self, name, loader=None, use_ssl=False, max_url_len=None):
"""Initialize, but do not start the WSGI server.
:param name: The name of the WSGI server given to the loader.
:param loader: Loads the WSGI application using the given name.
:returns: None
"""
self.name = name
# 这里的name就是WSGI server name，比如osapi_compute或者metadata
# nova.service's enabled_apis
self.binary = 'nova-%s' % name
self.topic = None
self.manager = self._get_manager()
self.loader = loader or wsgi.Loader()
# 从paste配置文件加载 Nova API 对应的 WSGI Application
self.app = self.loader.load_app(name)
# inherit all compute_api worker counts from osapi_compute
if name.startswith('openstack_compute_api'):
wname = 'osapi_compute'
else:
wname = name
self.host = getattr(CONF, '%s_listen' % name, "0.0.0.0")
self.port = getattr(CONF, '%s_listen_port' % name, 0)
self.workers = (getattr(CONF, '%s_workers' % wname, None) or
processutils.get_worker_count())
if self.workers and self.workers < 1:
worker_name = '%s_workers' % name
msg = (_("%(worker_name)s value of %(workers)s is invalid, "
"must be greater than 0") %
{'worker_name': worker_name,
'workers': str(self.workers)})
raise exception.InvalidInput(msg)
self.use_ssl = use_ssl
self.server = wsgi.Server(name,
self.app,
host=self.host,
port=self.port,
use_ssl=self.use_ssl,
max_url_len=max_url_len)
# Pull back actual port used
self.port = self.server.port
self.backdoor_port = None在nova-api运行过程中（发出了HTTP请求），Paste Deploy会将WSGI Server上监听到的HTTP请求根据Paste配置文件准确地路由到特定的WSGI Application，这其中经过了
nova.api.openstack.urlmap的urlmap_factory
nova.api.auth的pipeline_factory_v21
nova.api.openstack.compute的APIRouterV21.factory(路由到指定的app)
下面根据nova –debug list输出的请求信息，一一介绍
GET http://10.238.158.189:8774/v2.1/servers/detail
1.首先我们通过这个请求知道使用了v2.1的API，再根据paste配置文件中
[composite:osapi_compute] <--
use = call:nova.api.openstack.urlmap:urlmap_factory <--
/: oscomputeversions
/v2: openstack_compute_api_v21_legacy_v2_compatible
/v2.1: openstack_compute_api_v21 <--则WSGI服务器osapi_compute将监听到这个请求，根据第一行use的内容我们知道是由nova.api.openstack.urlmap模块的urlmap_factory进行分发，即对应了openstack_compute_api_v21
2.在paste配置文件中，又有
[composite:openstack_compute_api_v21] <--
use = call:nova.api.auth:pipeline_factory_v21 <--
noauth2 = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit noauth2 osapi_compute_app_v21
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21 <--可知又使用了nova.api.auth模块的pipeline_factory_v21进一步分发，并根据/etc/nova/nova.conf中认证策略“auth_strategy”选项使用参数keystone，在paste文件中，我们看到noauth2和keystone后面跟着一大串，不用过于深究，这是一个pipeline，最后一个是真正的app即osapi_compute_app_v21，从代码中分析可知根据这个pipeline，从后往前为这个app穿上一件件“外衣”，每一次封装都是封装成一个WSGI Application。
代码实现见nova/api/auth.py
def _load_pipeline(loader, pipeline):
filters = [loader.get_filter(n) for n in pipeline[:-1]]
app = loader.get_app(pipeline[-1])
filters.reverse()
for filter in filters:
app = filter(app)
return app3.再由paste配置文件中
[app:osapi_compute_app_v21]
paste.app_factory = nova.api.openstack.compute:APIRouterV21.factory需要调用nova.api.openstack.compute的APIRouterV21.factory
WSGI Application到具体的执行函数
这个部分是最复杂的部分，函数的调用和封装很复杂，有时候看到最深层次也看不大懂了，如果有愿意分享的小伙伴欢迎多多交流
APIRouterV21类的定义在nova/api/openstack/__init__.py中，主要完成对所有资源的加载和路由规则的创建（得到一个mapper包含所有资源的路由信息），APIRouterV21初始化的最后一步是将这个mapper交给它的父类nova.wsgi.Router，完成mapper和_dismatch()的关联，在openstack中，每个资源对应一个Controller，也对应一个WSGI Application，比如请求
GET http://10.238.158.189:8774/v2.1/servers/detail
对应的Contrller就是资源servers的Controller，具体位置是nova/api/openstack/compute/servers.py中的ServersController，这个Controller中定义了各种action，其中有一个函数是detail()，最终会调用这个函数得到想要的结果
下面详细介绍其过程
1.所有资源的加载以及路由规则的创建
class APIRouterV21(base_wsgi.Router):
def __init__(self, init_only=None):
def _check_load_extension(ext):
return self._register_extension(ext)
# 使用stevedore的EnabledExtensionManager类载入位于setup.cfg
# 的nova.api.v21.extensions的所有资源
self.api_extension_manager = stevedore.enabled.EnabledExtensionManager(
namespace=self.api_extension_namespace(),
check_func=_check_load_extension,
invoke_on_load=True,
invoke_kwds={"extension_info": self.loaded_extension_info})
mapper = ProjectMapper()
self.resources = {}
# NOTE(cyeoh) Core API support is rewritten as extensions
# but conceptually still have core
if list(self.api_extension_manager):
# NOTE(cyeoh): Stevedore raises an exception if there are
# no plugins detected. I wonder if this is a bug.
# 所有资源进行注册，同时使用mapper建立路由规则
self._register_resources_check_inherits(mapper)
# 扩展现有资源及其操作
self.api_extension_manager.map(self._register_controllers)
LOG.info(_LI("Loaded extensions: %s"),
sorted(self.loaded_extension_info.get_extensions().keys()))
super(APIRouterV21, self).__init__(mapper)2.父类Router将mapper和dispatch()关联起来
class Router(object):
"""WSGI middleware that maps incoming requests to WSGI apps."""
def __init__(self, mapper):
"""Create a router for the given routes.Mapper.
Each route in `mapper` must specify a 'controller', which is a
WSGI app to call. You'll probably want to specify an 'action' as
well and have your controller be an object that can route
the request to the action-specific method.
Examples:
mapper = routes.Mapper()
sc = ServerController()
# Explicit mapping of one route to a controller+action
mapper.connect(None, '/svrlist', controller=sc, action='list')
# Actions are all implicitly defined
mapper.resource('server', 'servers', controller=sc)
# Pointing to an arbitrary WSGI app. You can specify the
# {path_info:.*} parameter so the target app can be handed just that
# section of the URL.
mapper.connect(None, '/v1.0/{path_info:.*}', controller=BlogApp())
"""
self.map = mapper
self._router = routes.middleware.RoutesMiddleware(self._dispatch,
self.map) # <--
@webob.dec.wsgify(RequestClass=Request)
def __call__(self, req):
"""Route the incoming request to a controller based on self.map.
If no match, return a 404.
"""
return self._router
@staticmethod
@webob.dec.wsgify(RequestClass=Request)
def _dispatch(req):
"""Dispatch the request to the appropriate controller.
Called by self._router after matching the incoming request to a route
and putting the information into req.environ. Either returns 404
or the routed WSGI app's response.
"""
match = req.environ['wsgiorg.routing_args'][1]
if not match:
return webob.exc.HTTPNotFound()
app = match['controller']
return app这里的__call__函数进行了wsgify的封装，在这个Router被call的时候，或者是它的子类APIRouterV21在被call的时候，需要看webob.dec.wsgify里面是怎么进行封装的才能知道具体的执行过程。我们已经了解到，在从HTTP请求路由到特定的WSGI Apllication的时候，最终路由到的app是经过封装的，即经过了pipeline列出的一系列filter，所以这个时候我们要真正地调用时，需要一层层把外面的filter去掉，这个在/usr/local/lib/python2.7/dist-packages/webob/dec.py中的wsgify类中有一个__call__的定义
def __call__(self, req, *args, **kw):
"""Call this as a WSGI application or with a request"""
func = self.func
if func is None:
if args or kw:
raise TypeError(
"Unbound %s can only be called with the function it "
"will wrap" % self.__class__.__name__)
func = req
return self.clone(func)
if isinstance(req, dict):
if len(args) != 1 or kw:
raise TypeError(
"Calling %r as a WSGI app with the wrong signature")
environ = req
start_response = args[0]
req = self.RequestClass(environ)
req.response = req.ResponseClass()
try:
args = self.args
if self.middleware_wraps:
args = (self.middleware_wraps,) + args
resp = self.call_func(req, *args, **self.kwargs)
except HTTPException as exc:
resp = exc
if resp is None:
## FIXME: I'm not sure what this should be?
resp = req.response
if isinstance(resp, text_type):
resp = bytes_(resp, req.charset)
if isinstance(resp, bytes):
body = resp
resp = req.response
resp.write(body)
if resp is not req.response:
resp = req.response.merge_cookies(resp)
return resp(environ, start_response)
else:
if self.middleware_wraps:
args = (self.middleware_wraps,) + args
return self.func(req, *args, **kw)这个函数定义了，被dec.wsgify封装后的函数在被调用的时候，会不断地把外面的“外衣”脱掉，一直到最核心的app，这个app是一个直接接收参数（environ, start_response）的wsgi app，我们看到在Router类的__init__操作中，初始化了一个self.router = routes.middleware.RoutesMiddleware(self._dispatch,self.map)，而routes.middleware.RoutesMiddleware的__call_函数是一个标准的WSGI Application，它接收参数（environ, start_response），如下，
def __call__(self, environ, start_response):
"""Resolves the URL in PATH_INFO, and uses wsgi.routing_args
to pass on URL resolver results."""
old_method = None
if self.use_method_override:
req = None
#（以下省略...）所以在APIRouter被call的整个过程中，我们可以通过在dec.wsgify的__call__函数中加打印信息查看整个流程,打印每次被call时的func信息，结果如下（总共有三次类似的输出，因为有三次请求）
********************************************************************************
<bound method CORS.__call__ of <oslo_middleware.cors.CORS object at 0x7f15db89ef50>>
********************************************************************************
<bound method HTTPProxyToWSGI.__call__ of <oslo_middleware.http_proxy_to_wsgi.HTTPProxyToWSGI object at 0x7f15db89eed0>>
********************************************************************************
<bound method ComputeReqIdMiddleware.__call__ of <nova.api.compute_req_id.ComputeReqIdMiddleware object at 0x7f15db89ee50>>
********************************************************************************
<bound method FaultWrapper.__call__ of <nova.api.openstack.FaultWrapper object at 0x7f15db89ec50>>
********************************************************************************
<bound method RequestBodySizeLimiter.__call__ of <oslo_middleware.sizelimit.RequestBodySizeLimiter object at 0x7f15db89edd0>>
********************************************************************************
<bound method AuthProtocol.__call__ of <keystonemiddleware.auth_token.AuthProtocol object at 0x7f15db89e690>>
********************************************************************************
<bound method NovaKeystoneContext.__call__ of <nova.api.auth.NovaKeystoneContext object at 0x7f15db89e6d0>>
********************************************************************************
<bound method APIRouterV21.__call__ of <nova.api.openstack.compute.APIRouterV21 object at 0x7f15dbf45990>>和paste配置文件中的pipeline的顺序是一样的，即
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21小结
看了几天的代码，很多还是一头雾水，python调用的库很多，openstack里又经常进行封装操作，有时候一个调用栈非常深，很容易混乱，这里涉及到的比如mapper是怎么建立路由的，等等，都没有在此深究，这种先看官网的exsample再尝试看代码，多设断点多调试，调试技巧很重要。继续努力！有小伙伴感兴趣的欢迎讨论。
