基于Qinling和OpenStack的FaaS实验

2017-10-06 Lingxian Kong 更多博文 » 博客 » GitHub »

原文链接 https://lingxiankong.github.io/2017-10-06-qinling-openstack-integration.html
注:以下为加速网络访问所做的原文缓存,经过重新格式化,可能存在格式方面的问题,或偶有遗漏信息,请以原文为准。


之前介绍过 openstack 中对 notification 的处理,里面提及将 notification 与 Qinling 结合,可以实现类似于 AWS Lambda 与 S3结合的经典使用场景,今天我就一步一步的将我的实验过程记录下来,以备后用。

本实验的大致流程如下:

  • 用户在 qinling 创建 function,function 接收对象存储中的 container 和 object 作为参数,function 做的事情就是简单打印 object 的信息
  • 用户在 aodh 中创建 alarm,当收到 swift发来的对象上传成功的通知时,执行用户定义的 qinling function
  • 用户向 swift 中上传 object
  • 当 object 上传成功后,用户的 function 被自动触发

环境准备

本实验是在一个 devstack 环境中完成,我使用的 local.conf 文件内容如下:

[[local|localrc]]
RECLONE=False

enable_plugin qinling https://github.com/openstack/qinling
enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer.git
enable_plugin aodh https://git.openstack.org/openstack/aodh

LIBS_FROM_GIT+=python-qinlingclient
DATABASE_PASSWORD=password
ADMIN_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=password
RABBIT_PASSWORD=password
LOGFILE=$DEST/logs/stack.sh.log
VERBOSE=True
LOG_COLOR=False
SCREEN_LOGDIR=$DEST/logs
LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=1

ENABLED_SERVICES=rabbit,mysql,key

# Nova
ENABLED_SERVICES+=,n-api,n-cpu,n-cond,n-sch,n-novnc,n-cauth,n-api-meta,placement-api,placement-client

# Glance
ENABLED_SERVICES+=,g-api,g-reg

# Neutron
ENABLED_SERVICES+=,q-svc,q-dhcp,q-meta,q-agt,q-l3

# Ceilometer
CEILOMETER_BACKEND=gnocchi

# enable DVR
Q_PLUGIN=ml2
Q_ML2_TENANT_NETWORK_TYPE=vxlan
Q_DVR_MODE=dvr_snat

# Swift
ENABLED_SERVICES+=,swift
SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
SWIFT_REPLICAS=1

IMAGE_URLS+=",http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img"

服务配置

Swift

既然要接收 swift 的 notification,首先 swift 得支持对象上传成功后发送 notification,swift 默认没有这个能力,需要额外的 middleware 的支持。社区中有一个 CeilometerMiddleware,但该项目基本已停止更新,而且它的功能并不能满足实验需求。所以我对其中的 swift.py 文件做了少许改动,参见这里

因为在 swift.py 中需要引入 swift 包中的模块,会有命名冲突导致引入失败,所以需要做如下操作:

rm -f /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/swift.pyc
mv /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/swift.py /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/notify.py

使用我修改过的文件内容替换 /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/notify.py,然后修改/etc/swift/proxy-server.conf,配置 ceilometer middleware:

[pipeline:main]
pipeline = ... ceilometer proxy-server

[filter:ceilometer]
topic = notifications
driver = messagingv2
url = rabbit://stackrabbit:password@10.0.19.65:5672/
control_exchange = swift
paste.filter_factory = ceilometermiddleware.notify:filter_factory
set log_level = DEBUG

重启 swift proxy 服务:

systemctl restart devstack@s-proxy.service

Ceilometer

ceilometer 中的 notification agent 收到通知后,会先转换成 event,然后 publish 出去,所以要分别配置event_definitions.yamlevent_pipeline.yaml

首先在/etc/ceilometer/event_definitions.yaml末尾添加如下定义:

- event_type: objectstorage.object.upload
  traits: &objectstore_upload
    tenant_id:
      fields: payload.tenant_id
    container:
      fields: payload.container
    object:
      fields: payload.object

/etc/ceilometer/event_pipeline.yaml 的内容:

---
sources:
    - name: event_source
      events:
          - "*"
      sinks:
          - event_sink
sinks:
    - name: event_sink
      transformers:
      publishers:
          - gnocchi://
          - notifier://?topic=alarm.all

为了让 ceilometer 支持 notifier,要在/etc/ceilometer/ceilometer.conf中配置:

[publisher_notifier]
event_topic=event

重启 notification agent 服务:

systemctl restart devstack@ceilometer-anotification.service

Aodh

确保 aodh-listener 能够收到来自 ceilometer 的 event,在/etc/aodh/aodh.conf中配置:

[listener]
event_alarm_topic=alarm.all

因为需要 aodh 根据用户定义的 alarm 调用 qinling function,所以 aodh 中需要有调用 qinling 的 notifier,代码在这里,将代码拷贝到/opt/stack/aodh/aodh/notifier/qinling.py文件中,并在 aodh 的setup.cfg中配置:

aodh.notifier =
    ...
    trust+qinling = aodh.notifier.qinling:TrustQinlingFunctionNotifier

重新部署 aodh 代码:

cd /opt/stack/aodh
pip install -e .

重启 aodh 相关服务:

systemctl restart devstack@aodh-*
systemctl restart apache2.service

all-in-one

前面这么多手工步骤只是为了了解具体的修改细节,为了以后部署方便,我也写了一个简单的脚本:

rm -f /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/swift.py* /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/notify.py*
curl -s -S -L -o /usr/local/lib/python2.7/dist-packages/ceilometermiddleware/notify.py https://raw.githubusercontent.com/LingxianKong/qinling_utils/master/swift_ceilometermiddleware.py
sed -i '/driver = messaging/c driver = messagingv2' /etc/swift/proxy-server.conf
sed -i '/ceilometermiddleware/c paste.filter_factory = ceilometermiddleware.notify:filter_factory' /etc/swift/proxy-server.conf
systemctl restart devstack@s-proxy.service

cat <<EOF >> /etc/ceilometer/event_definitions.yaml
- event_type: objectstorage.object.upload
  traits: &objectstore_upload
    tenant_id:
      fields: payload.tenant_id
    container:
      fields: payload.container
    object:
      fields: payload.object
EOF
echo '          - notifier://?topic=alarm.all' >> /etc/ceilometer/event_pipeline.yaml
cat <<EOF >> /etc/ceilometer/ceilometer.conf
[publisher_notifier]
event_topic=event
EOF
systemctl restart devstack@ceilometer-anotification.service

cat <<EOF >> /etc/aodh/aodh.conf
[listener]
event_alarm_topic=alarm.all
EOF
curl -s -S -L -o /opt/stack/aodh/aodh/notifier/qinling.py https://raw.githubusercontent.com/LingxianKong/qinling_utils/master/aodh_qinlingnotifier.py
qinling_notifier='    trust+qinling = aodh.notifier.qinling:TrustQinlingFunctionNotifier'
sed -i "/trust+zaqar/ s/.*/&\n${qinling_notifier}/" /opt/stack/aodh/setup.cfg
pushd /opt/stack/aodh
pip install -e .
popd
systemctl restart devstack@aodh-*
systemctl restart apache2.service

实验步骤

配置好上述服务,具体的实验就很简单了。

# admin 租户创建 runtime
source openrc admin admin
openstack runtime create python openstackqinling/python-runtime

# demo 用户创建 function
cat <<EOF > ~/test_swift.py
import swiftclient
def stat_object(context, container, object):
    conn = swiftclient.Connection(
        session=context['os_session'],
        os_options={'region_name': 'RegionOne'},
    )
    obj_header = conn.head_object(container, object)
    return obj_header
EOF
openstack function create test_swift $RUNTIME_ID '{"source": "package"}' --entry test_swift.stat_object --file ~/test_swift.py

# demo 用户创建 alarm
aodh alarm create --name swift_alarm --type event --alarm-action trust+qinling://?function_id=$FUNCTION_ID --repeat-actions true --event-type objectstorage.object.*

# demo 用户上传 object
swift --os-auth-url http://10.0.19.65/identity/v3 --os-tenant-name demo --os-username demo --os-password password post lingxian
swift --os-auth-url http://10.0.19.65/identity/v3 --os-tenant-name demo --os-username demo --os-password password upload lingxian tox.ini

当 object 上传成功后,demo 用户可以使用 openstack function execution list 看到有一个新的 execution,并且可以看到 function 的输出:

$ openstack function execution list -f yaml
- Created_at: '2017-10-06 02:35:51'
  Description: null
  Function_id: 9be45c32-d12d-40df-97b8-5fbe12bd6f0d
  Id: a62ae49c-2e75-4f67-a850-cce4c34fce7f
  Input: '{"object": "tox.ini", "container": "lingxian"}'
  Output: '{"duration": 0.274, "output": {"content-length": "1783", "x-object-meta-mtime":
    "1506425620.843263", "accept-ranges": "bytes", "last-modified": "Fri, 06 Oct 2017
    02:35:45 GMT", "etag": "61e39be8895062f792989454a78f6b96", "x-timestamp": "1507257344.62552",
    "x-trans-id": "tx88ad4e9e0a3842068edfd-0059d6ec08", "date": "Fri, 06 Oct 2017
    02:35:52 GMT", "content-type": "application/octet-stream", "x-openstack-request-id":
    "tx88ad4e9e0a3842068edfd-0059d6ec08"}}'
  Status: success
  Sync: False
  Updated_at: '2017-10-06 02:35:52'