[Kolla簡介]
Kolla一開始為TripleO專案的一部分，但它跟TripleO的不同是，它是基於Docker container來deploy OpenStack。而Kolla於ocata版已拆分為kolla和kolla-ansible/kolla-k8s兩個部分:
- kolla: 用來建置production-ready images
- kolla-ansible/kolla-k8s: 用來部署containernized OpenStack
[如何產生kolla-build.conf來建置kolla映像檔]
- 安裝tox: sudo pip install tox
- 切換至kolla專案主目錄: cd kolla/
- 產生kolla-build.conf: tox -e genconfig #預設會放在etc/
- kolla-build.conf配置檔設定
[heat-base]
type = git
location = https://github.com/openstack/heat.git
reference = stable/pike
- include_header = ./.header # 自訂header檔案的位置
- include_footer = ./.footer # 自訂footer檔案的位置
[.header檔案內容]
ARG http_proxy=http://{proxy}:8080
ARG https_proxy=https://{https-proxy}:8080
ARG no_proxy=localhost
[.footer檔案內容]
ARG http_proxy=""
ARG https_proxy=""
ARG no_proxy=""
[Kolla映像檔建置]
- [方法一] python tools/build.py -b ubuntu {{服務}}
- 例如: sudo -E tools/build.py -b ubuntu glance neutron
- [方法二] kolla-build -b ubuntu {{服務}}
- kolla-build指令可以透過sudo pip install kolla/來安裝
[Kolla inventory]
* network_interface - While it is not used on its own, this provides the required default for other interfaces below.
* api_interface - This interface is used for the management network. The management network is the network OpenStack services uses to communicate to each other and the databases. There are known security risks here, so it’s recommended to make this network internal, not accessible from outside. Defaults to network_interface.
* kolla_external_vip_interface - This interface is public-facing one. It’s used when you want HAProxy public endpoints to be exposed in different network than internal ones. It is mandatory to set this option when kolla_enable_tls_external is set to yes. Defaults to network_interface.
* storage_interface - This is the interface that is used by virtual machines to communicate to Ceph. This can be heavily utilized so it’s recommended to put this network on 10Gig networking. Defaults to network_interface.
* cluster_interface - This is another interface used by Ceph. It’s used for data replication. It can be heavily utilized also and if it becomes a bottleneck it can affect data consistency and performance of whole cluster. Defaults to network_interface.
* tunnel_interface - This interface is used by Neutron for vm-to-vm traffic over tunneled networks (like VxLan). Defaults to network_interface.
* neutron_external_interface - This interface is required by Neutron. Neutron will put br-ex on it. It will be used for flat networking as well as tagged vlan networks. Has to be set separately.
* dns_interface - This interface is required by Designate and Bind9. Is used by public facing DNS requests and queries to bind9 and designate mDNS services. Defaults to network_interface.
* bifrost_network_interface - This interface is required by Bifrost. Is used to provision bare metal cloud hosts, require L2 connectivity with the bare metal cloud hosts in order to provide DHCP leases with PXE boot options. Defaults to network_interface.
- 注意如果interface是OVS bridge的話，介面卡代號記得從br-ethX置換成br_ethX (注意是底線)，詳細可以透過ansible -m setup -i {inventory} all去抓來看
[Kolla部署]
由於目前社群較多人使用的是kolla-ansible專案，所以以下將介紹kolla-ansible的部署指令
- 確認inventory
- 確認服務的ports是否被占用: tools/kolla-ansible prechecks
- 執行部署: sudo -E tools/kolla-ansible -i {{環境部署方式之inventory}} --configdir {{環境設定檔資料夾}} --password {{該環境使用密碼檔}} deploy
- Neutron網路配置在kolla-ansible專案中ansible/group_vars/all中修改
- kolla-ansible的role service一般在部署會經過register => config => bootstrap => restart service
- register: 註冊endpoint
- config: 配置
- bootstrap: migrate db
- restart service
[kolla-ansible各role的目錄結構]
bootstrap: 建立服務用的db，也就是執行*-manage db sync
bootstrap-service: 啟動服務用container，但會以前景模式啟動
handlers/main.yml : 規範要以背景常駐啟動的服務，主要設定檔及啟動參數都規範於此
- kolla已自訂撰寫模組kolla_toolbox、kolla_image 於ansible/library來動態載入
[kolla-ansible支援的操作actions]
- deploy
- precheck
- destroy
- reconfigure
- pull
[kolla-ansible重部署的方法]
- sudo docker ps -qa | sudo xargs docker rm -f #清除所有containers
- sudo docker volume ls -q | sudo xargs docker volume rm #清除docker volumes的所有資料
- sudo service docker restart # 重啟docker服務
[kolla-ansible資料庫修復]
- 當mariadb的galera抱怨連線有錯的時候可以下達tools/kolla-ansible -i {inventory} --configdir {configdir} --passwords {password } mariadb_revory
[kolla-ansible的腳本演進]
- 於ocata版本開始，將role的start任務轉交給handler處理，會於deploy.yml中透過meta: flush_handlers來執行
[kolla的HA]
[kolla的OVS]
- name: Ensuring OVS bridge is properly setup
command: docker exec openvswitch_db /usr/local/bin/kolla_ensure_openvswitch_configured {{ item.0 }} {{ item.1 }}
register: status
changed_when: status.stdout.find('changed') != -1
when:
- inventory_hostname in groups["network"]
or (inventory_hostname in groups["compute"] and computes_need_external_bridge | bool )
with_together:
- "{{ neutron_bridge_name.split(',') }}"
- "{{ neutron_external_interface.split(',') }}"
[kolla社群發展]
- Queens版將支援Ceph的Luminous版本，以支撐ceph fs以及ceph nfs等服務; Rocky有計畫要實現Ceph bluestore的功能
- 未來openstack-helm跟ansible-k8s專案有合併的趨勢
- 更多可參考
[kolla常見問題]
1. Hostname has to resolve to IP address of api_interface
TASK [prechecks : fail] ********************************************************************************************************************************************************
task path: /home/ubuntu/kolla/ansible/roles/prechecks/tasks/port_checks.yml:448
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: '{{ hostvars[item['item']]['ansible_' +
hostvars[item['item']]['api_interface']]['ipv4']['address'] }}' not in '{{ item.stdout }}'
fatal: [10.144.192.36]: FAILED! => {
"msg": "The conditional check ''{{ hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] }}' not in '{{ item.stdout }}'' failed. T he error was: Invalid conditional detected: EOL while scanning string literal (<unknown>, line 1)\n\nThe error appears to have been in '/home/ubuntu/kolla/ansible/roles/prechec ks/tasks/port_checks.yml': line 448, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- fail: msg=\" Hostname has to resolve to IP address of api_interface\"\n ^ here\n"
[ANS]
此問題常見於Kolla AIO部署上，請嘗試修改remote host上的/etc/hosts，確認被部署機器上api_interface所綁定的網路介面卡如eth0的ip跟hostname是可以互相解析的。
- 確認有127.0.0.1 localhost和<mgmt ip> <hostname>這兩行，若不行的話移除127.0.1.1 localhost看看
- 若有看到ansible的error，可確認ansible套件版本是否符合所需，N版以前建議ansible <= 2.2 、>=2.1 (目前實驗環境為2.1.6)。另外問了Kolla Reviewer，目前p和q版kolla的版本建議用<2.4的，目前coverage rate最高
2. inventory缺少的ansible供裝錯誤
TASK [nova : Ensuring config directories exist]
fatal: [10.144.192.36]: FAILED! =>
{"msg": "The conditional check 'inventory_hostname in groups[item.value.group]' failed.
The error was: error while evaluating conditional
(inventory_hostname in groups[item.value.group]): Unable to look up a name or
access an attribute in template string
({% if inventory_hostname in groups[item.value.group] %} True {% else %} False {% endif %}).\n
Make sure your variable name does not contain invalid characters like '-':
argument of type 'StrictUndefined' is not iterable\n
\nThe error appears to have been in '/home/ubuntu/kolla-ansible/ansible/roles/nova/tasks/config.yml':
line 13, column 3, but may\n
be elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:
\n\n\n- name: Ensuring config directories exist\n ^ here\n"}
to retry, use: --limit @/home/ubuntu/kolla-ansible/ansible/site.retry
[ANS] 檢查一下inventory的group是否有缺失，再來考慮是否是ansible版本問題
[參考資源]
