ARP、RARP、Proxy ARP

ARP

已知ip位址來查詢MAC位址

RARP(Reverse ARP)

已知MAC位址來反查IP位址

Proxy ARP

代理他人來回應其對應的MAC位址

Storm學習筆記

Storm的架構元件是基於:

1. Apache Zookeeper
   ---  Distributed System, used to store metadata
2. 0MQ
   ---  Async message transport-layer
3. Apache Thrift
   ---  Cross-Language Bridge, RPC
4. LMAX Disruptor
   ---  High performance queue shared by threads
    
5. Kryo
   ---  Serialization framework

• Spout(噴口):
        串流的來源, ex: 讀取log, API calls, event data, queues, ...
  
• Bolt(螺栓):
        串流的聚合(處理輸入串流並產生新的串流), ex: Stream joins, DBs, APIs, Filters, Aggregation,...
  
• Topology(拓樸):
         由Spout以及Bolt所構成的有向圖

Facebook Cover Photo Size

815px x 315px

http://www.google.com/imgres?imgurl=http://freeprofilecover.com/wp-content/uploads/2012/05/Facebook-timeline-dimensions-cover-app-icons-profile1.jpg&imgrefurl=http://freeprofilecover.com/what-is-the-facebook-cover-size-dimension&h=348&w=560&sz=101&tbnid=Q3y6qO66KzU9WM:&tbnh=76&tbnw=122&zoom=1&usg=__UBSR5Ngl9clO-GD8sl6HhMNZC3A=&docid=IC-0D5Tj26HaYM&sa=X&ei=TecNUo63NovzlAXZ4IG4CA&ved=0CEoQ9QEwAw&dur=358

MAE與RMSE的主要區別

RMSE >= MAE


The MAE and the RMSE can be used together to diagnose the variation in the errors in a set of forecasts. The RMSE will always be larger or equal to the MAE; the greater difference between them, the greater the variancein the individual errors in the sample. If the RMSE=MAE, then all the errors are of the same magnitude

Both the MAE and RMSE can range from 0 to ∞. They are negatively-oriented scores: Lower values are better.

http://www.eumetcal.org/resources/ukmeteocal/verification/www/english/msg/ver_cont_var/uos3/uos3_ko1.htm

L2 in L3 tunneling

為了解決vlan tag只能有4096個(2^12)的問題所以用此方法來解決

L2 in L3 tunneling to the rescue
You want a private L2 network between VMs
You already have working IP between the members of the network
Use that "underlay" IP network and overlay a private ethernet on top

SSH Tunnel

Reverse Tunneling

Unix/Linux: system load

load的就是一定时间内计算机有多少个active_tasks，也就是说是计算机的任务执行队列的长度，cpu计算的队列。


The comparative study of different load indices carried out by Ferrari et al.^[2] reported that CPU load information based upon the CPU queue length does much better in load balancing compared to CPU utilization
http://os.51cto.com/art/201012/240719.htm

http://en.wikipedia.org/wiki/Load_(computing)

bash for-counting-loop

在shell script中

for (( i=0 ; i<5; i++ ))
do
      # do something
done


這種for counting loop是在bash才有 ，sh沒有

Install Quantum on XenServer

For using the quantum plugin see: http://openvswitch.org/openstack/documentation/
You can:

• follow: [[[QuantumDevstack]]]
• ensure your VM network is not on a VLAN
• assuming you use Open vSwitch (default on XenServer 6.0.0 and later)
• run the quantum agent on Dom0
• add in nova.conf: xenapi_vif_driver=nova.virt.xenapi.vif.XenAPIOpenVswitchDriver

TODO - other steps not yet found.

ssh -i  identity-file(private-key)  

-i identity_file

Selects a file from which the identity (private key) for RSA or DSA authentication is read. The default is $HOME/.ssh/identity for protocol version 1, and $HOME/.ssh/id_rsa and $HOME/.ssh/id_dsa for protocol version 2. Identity files may also be specified on a per-host basis in the configuration file. It is possible to have multiple -i options (and multiple identities specified in configuration files).

PYTHON: range() vs xrange()

1. range 是全部產生完後，return一個 list 回來使用。
2. xrange 是一次產生一個值，並return一個值回來，所以xrange只適用於loop。


http://falldog7.blogspot.tw/2009/07/python-range-xrange.html

novncproxy套件安裝衝突解決辦法

dpkg -i --force-overwrite /var/cache/apt/archives/nova-novncproxy_1%3a2013.1-0ubuntu2~cloud1_all.deb

https://answers.launchpad.net/nova/+question/228890

AWS AMI/AKI/ARI之介紹

AMI (Amazon Machine Image) This is a virtual machine image in raw format, as
described above.
AKI (Amazon Kernel Image) A kernel file that the hypervisor will load initially to boot
the image. For a Linux machine, this would be a vmlinuz
file.
ARI (Amazon Ramdisk Image) An optional ramdisk file mounted at boot time. For a
Linux machine, this would be an initrd file.

[Quantum] quantum ovs-plugin agent的問題: br-int is not a bridge or a socket

https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/issues/75

https://wiki.openstack.org/wiki/ConfigureOpenvswitch

https://answers.launchpad.net/quantum/+question/181070

https://answers.launchpad.net/quantum/+question/205427

半虛擬化(para-virtualization) vs 全虛擬化(full-virtualization)

para-virtualization

full-virtualization

http://werdna1222coldcodes.blogspot.tw/2012/10/full-para-virtualization.html

Xenserver RRDs

Xenserver所使用的RRD Consolidation Function(CF)共有:
1. Average
2. Min
3. Max

There exist RRDs for individual VMs (including dom0) and the host. The VM RRDs are stored on the host on which they run, or the pool master when they are not running. This means that the location of VM must be known in order to retrieve the associated performance data.

DS Field Definitions:

DS	Defines a Data Source Field.
DS-Name	The name of this Data Source.
DST	Defines the Data Source Type. Can be GAUGE, COUNTER, DERIVE or ABSOLUTE.
HeartBeat	Defines the minimum heartbeat, the maximum number of seconds that can go by before a DS value is considered unknown.
Min	The minimum acceptable value. Values less than this number are considered unknown. This is optional.
Max	The maximum acceptable value. Values exceeding this number are considered unknown. This is optional.

DS欄位格式為"三文字，三數字"

ex:
DS:telnet:COUNTER:600:0:100000000
      (DS-Name) (DST) (HeartBeat)  (Min) (Max)

RRA

RRA means round robin archive. Defining an RRA is the tricky part!

This parameter describes how long you want to hold your data, and in what resolution. There can be more than on archive. If the time span of an archive is full, the data will be consolidated and saved into the next archive.

The format is

RRA:CF:xff:steps:rows

http://community.citrix.com/display/xs/Graphing+RRD+with+GNUPlot
http://community.citrix.com/display/xs/Getting+the+Last+RRD+Datapoints
http://community.citrix.com/display/xs/Using+XenServer+RRDs

mysql資料庫的授權(grant)

grant all privileges on '資料庫'.'表格名'  to ' 帳戶'@'主機'   identified by '密碼';

上傳tag到遠端的repo

上傳標籤到遠端

git push 並不會把標籤上傳到遠端，所以必須透過底下才行

$ git push origin v1.5
Counting objects: 50, done.
Compressing objects: 100% (38/38), done.
Writing objects: 100% (44/44), 4.56 KiB, done.
Total 44 (delta 18), reused 8 (delta 1)
To git@github.com:schacon/simplegit.git
* [new tag]         v1.5 -> v1.5

如果在本機端很多標籤，利用 –tags 一次上傳上去

$ git push origin --tags
Counting objects: 50, done.
Compressing objects: 100% (38/38), done.
Writing objects: 100% (44/44), 4.56 KiB, done.
Total 44 (delta 18), reused 8 (delta 1)
To git@github.com:schacon/simplegit.git
 * [new tag]         v0.1 -> v0.1
 * [new tag]         v1.2 -> v1.2
 * [new tag]         v1.4 -> v1.4
 * [new tag]         v1.4-lw -> v1.4-lw
 * [new tag]         v1.5 -> v1.5

http://blog.wu-boy.com/2010/11/git-%E7%89%88%E6%9C%AC%E6%8E%A7%E5%88%B6-%E5%A6%82%E4%BD%95%E4%BD%BF%E7%94%A8%E6%A8%99%E7%B1%A4tag/

Python基礎心法: list與dict的初始化宣告

dict

x  = {}

list 

x = []

指數平滑法

指數平滑法可以分成:
1. 一次指數平滑法
2. 二次指數平滑法
3. 三次指數平滑法

CA Server的種類

• Self-Signed CA
  亦可稱為 root CA。由於此種 CA 所發的憑證(certificate)是不經由任何上層 CA 所認證，而是以「自行認證」的方式進行認證。因此像是最上層的商業 CA，或是自行架設內部認證用的 CA，都可以屬於此類。
• Signed CA 
  不同於 Self-Signed CA，此種 CA 所發佈的憑證，可被上層的 CA 進行認證，而兩種 CA 的關係則是「Parent CA <==> Child CA」。 而通常設定上層 CA 時，除非是內部使用，不然使用商業 CA 是必須付費的!

http://godleon.blogspot.tw/2007/05/ca-certificate-certificate.html

ceilometer監控的資料類型

Type	Definition
Cumulative	Increasing over time (instance hours)
Gauge	Discrete items (floating IPs, image uploads) and fluctuating values (disk I/O)
Delta	Changing over time (bandwidth)

Linux系統附載均值

load average: 0.09, 0.05, 0.01

             (1min)(5min) (15min)

Reference

http://www.gracecode.com/posts/2973.html

STP: Spanning Tree Protocol

http://blog.xuite.net/alexx.huang/solution/33077662-STP+(Spanning+%E2%80%93+Tree+Protocol)+

RRDTools

RRDTools = Round-Robin Database Tools的縮寫

handle time-series data like network bandwidth,temperatures, CPU load, etc. The data are stored in a round-robin database (circular buffer), thus the system storage footprint remains constant over time.

Reference:

1. RRDtools tutorial, http://oss.oetiker.ch/rrdtool/tut/

OpenStack: Cloud Inventory Manager

CloudInventoryManager

https://wiki.openstack.org/wiki/CloudInventoryManager

[英文] provide的語法

provide X for 人

provide X with 東西

nova: virtapi

在nova中virtapi的定義放在/usr/share/pyshared/nova/virt/virtapi.py當中，只是用來制定api的spec

英文: 先行詞

先行詞是指句子中被一個跟在後面的代名詞所代替的字群；這個字群通常是個名詞及其修飾語。

http://www.taiwantestcentral.com/Grammar/Title.aspx?ID=236

OpenStack Xenserver Host Aggregate 建置筆記

所謂的Xenserver Host-Aggregte目的是為了讓Compute Node能利用Xensever Resource Pool的概念集結起來，在掛載像是NFS Share的SR以便做Live-migration或是VM-RESIZE。

1. 在XenServer Host上先製作出PV的Nova Compute Node + quantum-plugin-openvswitch-agent
    (ubuntu 12.04 LTS )
2. 然後再將要建置成resource pool的所有compute node中的/etc/nova/nova.conf加入以下的設定:
    sr_matching_filter =  default-sr:true

3. 再來利用aggregate-create在availability zone中建置邏輯資源池(logical resource pool)
    nova aggregate-create <pool-name>  <availability-zone>

4.  再來設定host-aggregate的metadata，告訴nova controller我要建置的是xenserver resource pool:
   nova aggregate-set-metadata <pool-id>  hypervisor_pool=true
   nova aggregate-set-metadata <pool-id>  operational_state=create

5.  從所有的計算節點當中選出一個當作Master ComputeNode:
  nova  aggregate-add-host <pool-id> <master-compute-node>

6. 其他後續加入的compute node會成為slave compute node:
nova aggregate-add-host <pool-id>  <slave-compute-node>



Reference:
1. http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-migrations.html#true-live-migration-kvm-libvirt

AMQP

http://lab.howie.tw/2012/07/whats-different-between-amqp-and-jms.html

Cluster的構成因子之一: Heartbeat

http://en.wikipedia.org/wiki/Heartbeat_(computing)

sFlow

http://flyfox.pixnet.net/blog/post/30971257-sflow

MySQL foreign key

http://webdesign.kerthis.com/sql/sql_foreign_key

OpenvSwitch Component Introduction

ovs−appctl − utility for configuring running Open vSwitch daemons ovs−controller − simple OpenFlow controller reference implementation ovs−dpctl − administer Open vSwitch datapaths

ovs−pki − OpenFlow public key infrastructure management utility

ovs−ofctl − administer OpenFlow switches ovs−vswitchd − Open vSwitch daemon

Nova: Xenserver Block Migration support on OpenStack

 Block migration  --- Xenserver必須支援Storage XenMotion的feature才可以使用此功能

Prerequisites 

• Compatible XenServer hypervisors. The hypervisors must support the Storage XenMotion feature. Please refer to the manual of your XenServer to make sure your edition has this feature.

	Note
	Please note, that you need to use an extra option --block-migrate for the live migration command, in order to use block migration.

	Note
	Please note, that block migration works only with EXT local storage SRs, and the server should not have any volumes attached.

Python: @property 把定義的函釋"當作"Attribute來查詢

http://joy2everyone.iteye.com/blog/910950

python的函式修飾器: decorator

Function Decorator

decorator用來主要是用來修飾函式而不必修改函式的本身。

只要利用@來接"修飾函式"就可以達到修飾效果了，這算是python的syntax sugar。

@ decorator_function

http://caterpillar.onlyfun.net/Gossip/Python/FunctionDecorator.html

SQL: reset AUTO_INCREMENT

ALTER TABLE `table_name` AUTO_INCREMENT=1;

程序執行的優先權: priority 與 nice

Cinder NFS Driver on OpenStack

在grizzly版本當中，cinder.volume.drivers.nfs.NfsDriver所製作出來的volumes就如同其他driver所製作出的一樣可以執行attach/detach的operation，但目前無法執行snapshot的運作。

OpenStack Security Group設定

為了讓Security Group的相關設定都交給Quantum Service來代理(proxy)處理，所以必須在/etc/nova/nova.conf中做以下的設定，以避免與原來Nova Networking所處理的Security Group相衝突。



## security groups ##
firewall_driver = nova.virt.firewall.NoopFirewallDriver
security_group_api = quantum

Kronos

主要是把XCP toolstack安裝到已經現有的OS環境裏頭


http://wiki.xen.org/wiki/Project_Kronos

QEMU和KVM的差異

QEMU和KVM大致上都是相同的，但QEMU不支援原生虛擬化(Native Virtualization)
he main difference is that QEMU does not support native virtualization. Consequently, QEMU has worse performance than KVM and is a poor choice for a production deployment.


http://docs.openstack.org/grizzly/openstack-compute/admin/content/qemu.html

File Sharing: NFS、CIFS、AutoFS

Reference:
http://itcat.pixnet.net/blog/post/17322013-%5Brhce%E8%AA%B2%E5%A0%82%E7%AD%86%E8%A8%98%5D-%E6%AA%94%E6%A1%88%E5%88%86%E4%BA%AB

OpenStack: nova.conf section

 Sections

Configuration options are grouped by section. The Compute config file supports the following sections.

[DEFAULT]

Almost all of the configuration options are organized into this section. If the documentation for a configuration option does not specify its section, assume that it should be placed in this one.

[cells]

The cells section is used for options for configuring cells functionality. See the Cells section of the OpenStack Compute Admin Manual for more details.

[baremetal]

This section is used for options that relate to the baremetal hypervisor driver.

[conductor]

The conductor section is used for options for configuring the nova-conductor service.

[trusted_computing]

The trusted_computing section is used for options that relate to the trusted computing pools functionality. Options in this section describe how to connect to a remote attestation service.

Nova Conductor Service: no-db-compute for compute node

Q: 為何要做nova conductor service?這個從folsom版本開始討論構思，一直到了grizzly版本才合併。nova conductor究竟是啥樣子的服務呢!!?

[Dev List]
     Nova Conductor Dev List, http://lists.openstack.org/pipermail/openstack-dev/2012-November/002573.html

A:
主要是不透過Nova DB的存取來降低效能的瓶頸以及compromised compute node會帶來淺在的危險。

 The nova-conductor service is key to completing no-db-compute. Conceptually, it implements a new layer on top of nova-compute.



[Objective]
1. Compute nodes are the least trusted part of a nova deployment, so removing direct database access is a step toward reducing the potential impact of a compromised compute node.

2. The other benefit of no-db-compute is for upgrades. Direct database access complicates the ability to do live rolling upgrades

Reference:

1. http://cloudystuffhappens.blogspot.tw/2013/04/understanding-nova-conductor-in.html
2. http://russellbryantnet.wordpress.com/2013/02/19/deployment-considerations-for-nova-conductor-service-in-openstack-grizzly/

Cinder: volume-type

What's Volume Type?
In the context of Cinder, Volume Type is an abstraction of various properties of a volume. It is _NOT_ type of back-end storage although one can create one volume type that only certain back-end is able to serve.

In current Cinder's implementation, volume type is just a string/name: 0

mysql> desc volume_types;
+------------+--------------+------+-----+---------+----------------+
| Field      | Type         | Null | Key | Default | Extra          |
+------------+--------------+------+-----+---------+-------------
| created_at | datetime     | YES  |     | NULL    |                |
| updated_at | datetime     | YES  |     | NULL    |                |
| deleted_at | datetime     | YES  |     | NULL    |                |
| deleted    | tinyint(1)   | YES  |     | NULL    |                |
| id         | int(11)      | NO   | PRI | NULL    | auto_increment |
| name       | varchar(255) | YES  |     | NULL    |                |
+------------+--------------+------+-----+---------+----------------+

https://etherpad.openstack.org/grizzly-cinder-volumetypes

Python: List

http://blog.eddie.com.tw/2011/10/13/python-list/

What's co-routine

Coroutines are computer program components that generalize subroutines to allow multiple entry points for suspending and resuming execution at certain locations. Coroutines are well-suited for implementing more familiar program components such as cooperative tasks,iterators, infinite lists and pipes.

Reference:

1. coroutine wiki, http://en.wikipedia.org/wiki/Coroutine

git: 改變commit(送交)的歷史

如果要改變最近一次的commit紀錄的話，則可以下達:

   git commit --amend



Reference:
1. http://git-scm.com/book/zh-tw/Git-%E5%B7%A5%E5%85%B7-%E9%87%8D%E5%AF%AB%E6%AD%B7%E5%8F%B2

Python: import modules from remote server

http://code.activestate.com/recipes/305277-import-modules-from-a-remote-server/

Python: coroutine

http://blog.ez2learn.com/2010/07/17/talk-about-coroutine-and-gevent/

Python: understanding python super() and __init__() methods

Super lets you avoid referring to the base class explicitly, which can be nice. But the main advantage comes with multiple inheritance, where all sorts of fun stuff can happen. See the standard docs on superif you haven't already.




Reference:
1. http://stackoverflow.com/questions/576169/understanding-python-super-and-init-methods

Linux: rpcinfo

rpcinfo -t(tcp)  <Remote-Server-IP-Address>

Citrix Xenserver: td-util

/usr/sbin/td-util

tapdisk utility

pickle: python object serialization

http://docs.python.org/2/library/pickle.html

MVC=Model-View-Control

M=Model, 負責資料儲存
V=View, 負責資料的呈現，例如網頁。
C=Control, 負責程式的邏輯控制

定義好framework的話則可將一些程式實作上的細節給隱藏起來(encapsulate)加以抽象化，再降低程式之間的隅合程度。

再以資料庫查詢應用為例，使用 MVC 而未使用 framework 之前，我們會用 sample1.php 的寫法。

sample1.php

1 2 3 4 5 6 7 8 9 10 11 12 13 14

<?php require_once 'config.php'; // get database address, user and password $dbh = new PDO(DB_CONNECT_STRING, DB_USER, DB_PASSWORD); function getFoo() {     global $dbh;     $fooSet = array();     $i = 0;     foreach ($dbh->query('SELECT * from FOO') as $row) {         $fooSet[$i++] = $row;     }     return $fooSet; } ?>

接著，假設我們使用了某個名為 "MyCommonApp" 的 framework 之後，按該 framework 的預先建置內容，我們改成 sample2.php 的寫法。

sample2.php

1 2 3 4 5 6 7 8 9 10 11 12 13 14

<?php require_once 'MyCommonApp.php'; class FooApp extends MyCommonApp {     function getFoo() {         $fooSet = array();         $i = 0;         foreach ($this->dbh->query('SELECT * from FOO') as $row) {             $fooSet[$i++] = $row;         }         return $fooSet;     } } ?>

sample1.php 和 sample2.php 兩種寫法的差異，一眼可見資料庫連線的動作被隱蔽了，而且也不需要用全域變數 (global) $dbh ， $dbh 變成了個體成員 $this->dbh 。更進一步，被隱蔽的部份可以加以抽象化，如 MyCommonApp 可能會支援多種資料庫系統，或是支援不同的資料庫連接層 (DBA, MDB2 等) 。但這些細節皆被 framework 隱蔽起來， programmer 只需要專注 class FooApp 的內容即可。還有一件事要提，上述內容沒有 View 和 Control 的程式碼，因為根本不需要知道。 View 只管呈現陣列 ($fooSet) 的內容，不管是 sample1.php 或是 sample2.php ，只要 Model 丟出來的是個陣列， View 就不用作任何修改。這也是 MVC 架構所帶來的好處：低耦合度，各項工作間彼此獨立，其中一個項目的修改動作不會連帶要修改其他項目。在 team-work 中也意味著三種設計工作可以交由三個人同時進行。

http://blog.roodo.com/rocksaying/archives/2617123.html