ubuntu 用apt-get remove 失效的package

使用大招
以flashplugin-nonfree為例

cd /var/lib/dpkg/info
sudo rm flashplugin-nonfree.*

然後

sudo dpkg --remove --force-remove-reinstreq flashplugin-nonfree
Posted in hosting | Leave a comment

Modern Python Hacker 必須要會的3把刀

衍伸:

相對於 ruby:

  • RVM
  • Bundler
  • Capistrano
Posted in programming | Tagged | Leave a comment

什麼是 Pythonnic ?

試以python實作以下C語言

for (i=0; i < mylist_length; i++) {
 
   do_something(mylist[i]);
 
}

新手python會這樣子寫:

i = 0
 
while i < mylist_length:
 
   do_something(mylist[i])
 
   i += 1

稍微中手會這樣子寫:

for i in range(mylist_length):
   do_something(mylist[i])

真正了解Pythonnic 會這樣子寫:

for element in mylist:
   do_something(element)

另一個例子
C:

void foo(int* a, float* b) {
 
    *a = 3;
    *b = 5.5;
 
}
 
int alpha;
int beta;
foo(&alpha, &beta);

python:
What the hell!!?

def foo(a, b):
    a[0] = 3
    b[0] = 5.5
 
alpha = [0]
beta = [0]
foo(alpha, beta)
alpha = alpha[0]
beta = beta[0]

正解!

def foo():
    return 3, 5.5
 
alpha, beta = foo()
Posted in programming | Tagged | Leave a comment

彩蛋 Python 的禪道

在python 當你

import this

就會出現python的禪道

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

還滿有趣的

以下為中譯
from: http://wiki.python.org.tw/The%20Zen%20Of%20Python

美麗優於醜陋,明講好過暗諭。

簡潔者為上,複雜者次之,繁澀者為下。

平舖善於層疊,勻散勝過稠密;以致輕鬆易讀。

特例難免但不可打破原則,務求純淨卻不可不切實際。

斷勿使錯誤靜靜流逝,除非有意如此。

在模擬兩可之間,拒絕猜測的誘惑。

總會有一種明確的寫法,最好也只有一種,

但或須細想方可得。

凡事雖應三思後行,但坐而言不如起而行。

難以解釋的實作方式,必定是壞方法。

容易解釋的實作方式,可能是好主意。

命名空間讚,吾人多實用。

Posted in programming | Tagged | Leave a comment

git flow

A successful Git branching model:
http://nvie.com/posts/a-successful-git-branching-model/
software:
https://github.com/nvie/gitflow
說明:
http://jeffkreeftmeijer.com/2010/why-arent-you-using-git-flow/

Posted in programming | Tagged | Leave a comment

MySQL Cluster

Posted in hosting | Tagged | Leave a comment

Use HAProxy on MySQL Cluster

HAProxy 原本是用在http網站分流的,
但是只要開啟tcp mode就可以作為MySQL的Load Balancer。

之前的設定好MySQL Cluster 大軍如下

Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=1    @vm1ip  (mysql-5.1.41 ndb-7.0.13, Nodegroup: 0)
id=2    @vm2ip  (mysql-5.1.41 ndb-7.0.13, Nodegroup: 0, Master)
 
[ndb_mgmd(MGM)] 1 node(s)
id=10   @vm3ip  (mysql-5.1.41 ndb-7.0.13)
 
[mysqld(API)]   2 node(s)
id=20   @vm4ip  (mysql-5.1.41 ndb-7.0.13)
id=21   @vm5ip  (mysql-5.1.41 ndb-7.0.13)

由於開啟了MySQL Cluster中兩個以上的API node (或稱SQL node),
所以加了一台HAPorxy作為API node的統一入口點

安裝好haporxy後 haproxy.cfg設定檔如下

listen MySQL 0.0.0.0:5000
retries 3
mode tcp
option mysql-check haproxy
balance roundrobin
option tcplog
server mysql1 vm4ip check port 5000
server mysql1 vm5ip check port 5000 backup

由於指定了mysql-check 的user 所以必須在兩個API node各執行以下SQL,方便haproxy進入查看

USE mysql;
INSERT INTO USER (Host,USER) VALUES ('','haproxy');
FLUSH PRIVILEGES;

另外如果想要看haproxy的log
則必須在/etc/rsyslog.d/haproxy.conf 加上

$ModLoad imudp
$UDPServerRun 514
$UDPServerAddress 127.0.0.1
local0.* -/var/log/haproxy_0.log
local1.* -/var/log/haproxy_1.log

然後重起一下rsyslog

restart rsyslog

接著就可以看haproxy有沒有正常的跑起來

tail -f /var/log/haproxy*.log

別忘了設定一下logrotate /etc/logrotate.d/haproxy

/var/log/haproxy*.log
{
rotate 4
weekly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
reload rsyslog &gt;/dev/null 2&gt;&amp;1 || true
endscript
}
Posted in hosting | Tagged , | Leave a comment

MySQL Cluster Foreign key Not ignored

最近在deploy Django在MySQL Cluster的時候遇到一個問題,
就是MySQL Cluster 用的DB engine 是 NDB,
ndb跟MyISAM基本上是不支援Foreign key,目前只有InnoDB才支援。
詳見:
http://dev.mysql.com/doc/refman/5.1/en/storage-engines.html
但是最新的MySQL Cluster版本: 7.1.15a
在deploy Django的時候出現 Foreign key clause is not yet supported in conjunction with partitioning
導致deploy失敗,
但是手冊上說明
http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-limitations-unsupported.html
以及bug#58929描述
http://bugs.mysql.com/bug.php?id=58929
當MySQL 用NDB實遇到Foreign key的SQL command 應該是略過而不出現錯誤
(行為跟MyISAM)
於是我把MySQL Cluster版本dwongrade到7.0.13就可以成功deploy Django的schema了,算是暫時解了這個困境。

但是這事衍生出一件滿重大的事就是:

不要再依賴Foreign key來靠Database綁住DB關聯性了,應該是靠程式自己去控管DB關聯性,否則就會被綁死在InnoDB上。

將來
如果想要加強可靠性用MySQL Cluster -> NDB不支援
要加強速度用Memory -> MySQL Memory不支援
更別說目前很火紅的NoSQL,當然也不支援

這樣就得回頭去改程式了

Posted in hosting | Tagged | Leave a comment

OpenStack running MySQL Cluster

OpenStack:

- Hardware : Dell OPTIPLEX 780 2U 4G ram
- OpenStack Verison: Diablo Release
- Hypervisor OS: ubuntu 11.04
- Install script: https://github.com/jsleetw/OpenStack-NOVA-Installer-Script/blob/master/nova-CC-install-v1.1.sh
- Config Glance:
Add config to /etc/nova.conf

--glance_host=0.0.0.0
--glance_port=9292
--image_service=nova.image.glance.GlanceImageService

- Add fireware rule for MySQL Cluster

euca-authorize -P icmp -t -1:-1 default 
euca-authorize -P tcp -p 22 default
euca-authorize -P tcp -p 1186 default
euca-authorize -P tcp -p 2202 default

MySQL Cluster:
- Start 3 x m1.small(2G ram) INSTANCE running MySQL Cluster (on Ubuntu 11.04)
– 1 Mgmt Node 2 Data Node 1 SQL Node
- MySQL Cluster Version: http://www.mysql.com/downloads/cluster/

- config.ini:

[ndb_mgmd]
hostname=vm1
datadir=/home/user/my_cluster/ndb_data
[ndbd default]
noofreplicas=2
datadir=/home/user/my_cluster/ndb_data
ServerPort= 2202
[ndbd]
hostname=vm2
[ndbd]
hostname=vm3

- my.cnf:

[mysqld]
ndbcluster
datadir=/home/user1/my_cluster/mysqld_data
basedir=/home/user1/mysqlc
port=5000

- start MySQL Cluster command
Start up Mgmt Node on vm1

cd mysqlc
scripts/mysql_install_db --no-defaults --datadir=$HOME/my_cluster/mysqld_data/
$HOME/mysqlc/bin/ndb_mgmd -f conf/config.ini --initial --configdir=$HOME/my_cluster/conf/

Start data node on vm2/vm3

$HOME/mysqlc/bin/ndbd --initial -c vm1:1186

check status on vm1

$HOME/mysqlc/bin/ndb_mgm -e show

When data node sync finish start SQL node on vm1

$HOME/mysqlc/bin/mysqld --defaults-file=conf/my.cnf &

finish!

reference : http://downloads.mysql.com/tutorials/cluster/GetMySQLClusterRunning-LINUX.pdf

Posted in hosting | Tagged , | Leave a comment

在Linode 512上的 php5-fpm.conf 設定

目前使用nginx + php-fpm 跑在ram只有512mb的Linode上
使用ab反覆壓力測試

ab -k -c 1010 -n 100000 url

發現將原本php5-fpm.conf

pm = static
pm.max_children = 10

改為

pm = dynamic
pm.max_children = 10
pm.start_servers = 2
pm.max_spare_servers = 2

會有妙不可言的效果!

Posted in hosting | Tagged , | 3 Comments