LinodeでInnodbの設定ファイルを考える
概要
Linodeで自作WEBアプリを公開していたりしますが、mysqldを立ち上げることがなかったので(参照メインなのでmemcached+SQLiteで足りていた)あんまりmy.cnfのパラメータを考えてなかったんですがちょっと考えてみたという話です。
設定ファイル
my.cnfはこんな感じです。正直ここをほぼそのまま流用してます。
この記事ではメモリは256MBの環境の設定です。
http://d.hatena.ne.jp/sh2/20100415
[mysqld] character_set_server = utf8 collation_server = utf8_general_ci #transaction_isolation = READ-COMMITTED ignore-builtin-innodb plugin-load = innodb=ha_innodb_plugin.so;innodb_trx=ha_innodb_plugin.so;innodb_locks=ha_innodb_plugin.so;innodb_lock_waits=ha_innodb_plugin.so;innodb_cmp=ha_innodb_plugin.so;innodb_cmp_reset=ha_innodb_plugin.so;innodb_cmpmem=ha_innodb_plugin.so;innodb_cmpmem_reset=ha_innodb_plugin.so default_storage_engine = InnoDB innodb_file_per_table = 1 innodb_flush_log_at_trx_commit = 1 innodb_buffer_pool_size = 4M innodb_log_buffer_size = 1M #innodb_log_file_size = 128M #server_id = 1 #log_bin = mysql-bin #binlog_format = MIXED #binlog_cache_size = 128K #sync_binlog = 1 slow_query_log = 1 long_query_time = 0.1 query_cache_type = 1 query_cache_size = 1M key_buffer_size = 512K sort_buffer_size = 512K read_buffer_size = 128K max_allowed_packet = 16M max_connections = 64 thread_cache_size = 8 table_open_cache = 4096 [client] default_character_set = utf8 port = 3306 socket = /var/lib/mysql/mysql.sock
mysqld起動
こんな感じ。ほとんどメモリ使ってません。動いてないからですが。
% free -m total used free shared buffers cached Mem: 498 467 30 0 19 406 -/+ buffers/cache: 41 456 Swap: 255 1 254 % top PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10299 mysql 20 0 99248 9496 4188 S 0.0 1.9 0:00.02 mysqld
MySQL Benchmark Suite
時間かかるのでひとまずinsertのみを測定します。なんですが、このinsertがまたとてつもなく長いのです。
% sudo ./test-insert --user=root --password=XXXX
結果
Testing server 'MySQL 5.1.49 log' at 2010-08-02 21:15:25 Testing the speed of inserting data into 1 table and do some selects on it. The tests are done with a table that has 100000 rows. Generating random keys Creating tables Inserting 100000 rows in order Inserting 100000 rows in reverse order Inserting 100000 rows in random order Time for insert (300000): 126 wallclock secs ( 0.70 usr 15.02 sys + 0.00 cusr 0.00 csys = 15.72 CPU) Testing insert of duplicates Time for insert_duplicates (100000): 20 wallclock secs ( 3.68 usr 1.95 sys + 0.00 cusr 0.00 csys = 5.63 CPU) Test of prepared+execute/once prepared many execute selects Time for prepared_select (100000): 35 wallclock secs (11.74 usr 1.79 sys + 0.00 cusr 0.00 csys = 13.53 CPU) Time for once_prepared_select (100000): 31 wallclock secs ( 5.56 usr 3.25 sys + 0.00 cusr 0.00 csys = 8.81 CPU) Retrieving data from the table Time for select_big (10:3000000): 12 wallclock secs (10.97 usr 0.00 sys + 0.00 cusr 0.00 csys = 10.97 CPU) Time for order_by_big_key (10:3000000): 13 wallclock secs (10.84 usr 0.00 sys + 0.00 cusr 0.00 csys = 10.84 CPU) Time for order_by_big_key_desc (10:3000000): 13 wallclock secs (10.22 usr 0.00 sys + 0.00 cusr 0.00 csys = 10.22 CPU) Time for order_by_big_key_prefix (10:3000000): 13 wallclock secs ( 3.93 usr 5.23 sys + 0.00 cusr 0.00 csys = 9.16 CPU) Time for order_by_big_key2 (10:3000000): 12 wallclock secs ( 0.00 usr 9.77 sys + 0.00 cusr 0.00 csys = 9.77 CPU) Time for order_by_big_key_diff (10:3000000): 12 wallclock secs ( 0.00 usr 10.21 sys + 0.00 cusr 0.00 csys = 10.21 CPU) Time for order_by_big (10:3000000): 14 wallclock secs ( 0.00 usr 10.41 sys + 0.00 cusr 0.00 csys = 10.41 CPU) Time for order_by_range (500:125750): 1 wallclock secs ( 0.00 usr 0.53 sys + 0.00 cusr 0.00 csys = 0.53 CPU) Time for order_by_key_prefix (500:125750): 0 wallclock secs ( 0.00 usr 0.52 sys + 0.00 cusr 0.00 csys = 0.52 CPU) Time for order_by_key2_diff (500:250500): 2 wallclock secs ( 0.00 usr 0.97 sys + 0.00 cusr 0.00 csys = 0.97 CPU) Time for select_diff_key (500:1000): 0 wallclock secs ( 0.00 usr 0.09 sys + 0.00 cusr 0.00 csys = 0.09 CPU) Time for select_range_prefix (5010:42084): 1 wallclock secs ( 0.00 usr 1.13 sys + 0.00 cusr 0.00 csys = 1.13 CPU) Time for select_range_key2 (5010:42084): 2 wallclock secs ( 0.00 usr 1.10 sys + 0.00 cusr 0.00 csys = 1.10 CPU) Time for select_key_prefix (200000): 66 wallclock secs ( 0.00 usr 28.50 sys + 0.00 cusr 0.00 csys = 28.50 CPU) Time for select_key (200000): 65 wallclock secs ( 0.00 usr 29.32 sys + 0.00 cusr 0.00 csys = 29.32 CPU) Time for select_key_return_key (200000): 62 wallclock secs ( 0.00 usr 28.33 sys + 0.00 cusr 0.00 csys = 28.33 CPU) Time for select_key2 (200000): 70 wallclock secs ( 0.00 usr 28.75 sys + 0.00 cusr 0.00 csys = 28.75 CPU) Time for select_key2_return_key (200000): 63 wallclock secs ( 0.00 usr 28.92 sys + 0.00 cusr 0.00 csys = 28.92 CPU) Time for select_key2_return_prim (200000): 62 wallclock secs ( 0.00 usr 28.08 sys + 0.00 cusr 0.00 csys = 28.08 CPU) Test of compares with simple ranges Time for select_range_prefix (20000:43500): 2 wallclock secs ( 0.00 usr 1.33 sys + 0.00 cusr 0.00 csys = 1.33 CPU) Time for select_range_key2 (20000:43500): 2 wallclock secs ( 0.00 usr 1.36 sys + 0.00 cusr 0.00 csys = 1.36 CPU) Time for select_group (111): 1 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Time for min_max_on_key (15000): 4 wallclock secs ( 0.00 usr 1.90 sys + 0.00 cusr 0.00 csys = 1.90 CPU) Time for min_max (60): 7 wallclock secs ( 0.00 usr 0.01 sys + 0.00 cusr 0.00 csys = 0.01 CPU) Time for count_on_key (100): 11 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Time for count (100): 21 wallclock secs ( 0.00 usr 0.03 sys + 0.00 cusr 0.00 csys = 0.03 CPU) Time for count_distinct_big (20): 3 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing update of keys with functions Time for update_of_key (50000): 52 wallclock secs ( 0.00 usr 2.71 sys + 0.00 cusr 0.00 csys = 2.71 CPU) Time for update_of_key_big (501): 35 wallclock secs ( 0.00 usr 0.03 sys + 0.00 cusr 0.00 csys = 0.03 CPU) Testing update with key Time for update_with_key (300000): 121 wallclock secs ( 0.00 usr 15.63 sys + 0.00 cusr 0.00 csys = 15.63 CPU) Time for update_with_key_prefix (100000): 41 wallclock secs ( 0.00 usr 7.10 sys + 0.00 cusr 0.00 csys = 7.10 CPU) Testing update of all rows Time for update_big (10): 35 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing left outer join Time for outer_join_on_key (10:10): 2 wallclock secs ( 0.00 usr 0.01 sys + 0.00 cusr 0.00 csys = 0.01 CPU) Time for outer_join (10:10): 3 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for outer_join_found (10:10): 2 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for outer_join_not_found (500:10): 2 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing SELECT ... WHERE id in (10 values) Time for select_in (500:5000) 0 wallclock secs ( 0.00 usr 0.12 sys + 0.00 cusr 0.00 csys = 0.12 CPU) Time for select_join_in (500:5000) 1 wallclock secs ( 0.00 usr 0.10 sys + 0.00 cusr 0.00 csys = 0.10 CPU) Testing SELECT ... WHERE id in (100 values) Time for select_in (500:50000) 0 wallclock secs ( 0.00 usr 0.27 sys + 0.00 cusr 0.00 csys = 0.27 CPU) Time for select_join_in (500:50000) 0 wallclock secs ( 0.00 usr 0.21 sys + 0.00 cusr 0.00 csys = 0.21 CPU) Testing SELECT ... WHERE id in (1000 values) Time for select_in (500:500000) 2 wallclock secs ( 0.00 usr 1.86 sys + 0.00 cusr 0.00 csys = 1.86 CPU) Time for select_join_in (500:500000) 2 wallclock secs ( 0.00 usr 1.79 sys + 0.00 cusr 0.00 csys = 1.79 CPU) Testing INSERT INTO ... SELECT Time for insert_select_1_key (1): 3 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for insert_select_2_keys (1): 4 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for drop table(2): 0 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing delete Time for delete_key (10000): 7 wallclock secs ( 0.00 usr 0.50 sys + 0.00 cusr 0.00 csys = 0.50 CPU) Time for delete_range (12): 5 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Insert into table with 16 keys and with a primary key with 16 parts Time for insert_key (100000): 1410 wallclock secs ( 0.00 usr 7.80 sys + 0.00 cusr 0.00 csys = 7.80 CPU) Testing update of keys Time for update_of_primary_key_many_keys (256): 6541 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Deleting rows from the table Time for delete_big_many_keys (128): 4410 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Deleting everything from table Time for delete_all_many_keys (1): 4410 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Inserting 100000 rows with multiple values Time for multiple_value_insert (100000): 4 wallclock secs ( 0.00 usr 0.25 sys + 0.00 cusr 0.00 csys = 0.25 CPU) Time for drop table(1): 0 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Total time: 13429 wallclock secs (57.64 usr 277.00 sys + 0.00 cusr 0.00 csys = 334.64 CPU) [homepage@li85-220]/usr/share/sql-bench% sudo ./test-insert --user=root --password=v6isJhM1 [sudo] password for homepage: Testing server 'MySQL 5.1.49 log' at 2010-08-03 9:48:27 Testing the speed of inserting data into 1 table and do some selects on it. The tests are done with a table that has 100000 rows. Generating random keys Creating tables Inserting 100000 rows in order Inserting 100000 rows in reverse order Inserting 100000 rows in random order Time for insert (300000): 129 wallclock secs ( 0.74 usr 14.95 sys + 0.00 cusr 0.00 csys = 15.69 CPU) Testing insert of duplicates Time for insert_duplicates (100000): 18 wallclock secs ( 3.54 usr 1.95 sys + 0.00 cusr 0.00 csys = 5.49 CPU) Test of prepared+execute/once prepared many execute selects Time for prepared_select (100000): 37 wallclock secs (12.27 usr 1.87 sys + 0.00 cusr 0.00 csys = 14.14 CPU) Time for once_prepared_select (100000): 32 wallclock secs ( 5.57 usr 3.43 sys + 0.00 cusr 0.00 csys = 9.00 CPU) Retrieving data from the table Time for select_big (10:3000000): 11 wallclock secs (10.77 usr 0.00 sys + 0.00 cusr 0.00 csys = 10.77 CPU) Time for order_by_big_key (10:3000000): 14 wallclock secs (11.27 usr 0.00 sys + 0.00 cusr 0.00 csys = 11.27 CPU) Time for order_by_big_key_desc (10:3000000): 14 wallclock secs (10.96 usr 0.00 sys + 0.00 cusr 0.00 csys = 10.96 CPU) Time for order_by_big_key_prefix (10:3000000): 13 wallclock secs ( 1.90 usr 7.14 sys + 0.00 cusr 0.00 csys = 9.04 CPU) Time for order_by_big_key2 (10:3000000): 10 wallclock secs ( 0.00 usr 9.03 sys + 0.00 cusr 0.00 csys = 9.03 CPU) Time for order_by_big_key_diff (10:3000000): 12 wallclock secs ( 0.00 usr 9.77 sys + 0.00 cusr 0.00 csys = 9.77 CPU) Time for order_by_big (10:3000000): 14 wallclock secs ( 0.00 usr 10.79 sys + 0.00 cusr 0.00 csys = 10.79 CPU) Time for order_by_range (500:125750): 1 wallclock secs ( 0.00 usr 0.51 sys + 0.00 cusr 0.00 csys = 0.51 CPU) Time for order_by_key_prefix (500:125750): 1 wallclock secs ( 0.00 usr 0.43 sys + 0.00 cusr 0.00 csys = 0.43 CPU) Time for order_by_key2_diff (500:250500): 1 wallclock secs ( 0.00 usr 0.82 sys + 0.00 cusr 0.00 csys = 0.82 CPU) Time for select_diff_key (500:1000): 0 wallclock secs ( 0.00 usr 0.09 sys + 0.00 cusr 0.00 csys = 0.09 CPU) Time for select_range_prefix (5010:42084): 2 wallclock secs ( 0.00 usr 1.17 sys + 0.00 cusr 0.00 csys = 1.17 CPU) Time for select_range_key2 (5010:42084): 1 wallclock secs ( 0.00 usr 1.17 sys + 0.00 cusr 0.00 csys = 1.17 CPU) Time for select_key_prefix (200000): 66 wallclock secs ( 0.00 usr 28.17 sys + 0.00 cusr 0.00 csys = 28.17 CPU) Time for select_key (200000): 64 wallclock secs ( 0.00 usr 29.20 sys + 0.00 cusr 0.00 csys = 29.20 CPU) Time for select_key_return_key (200000): 61 wallclock secs ( 0.00 usr 28.27 sys + 0.00 cusr 0.00 csys = 28.27 CPU) Time for select_key2 (200000): 70 wallclock secs ( 0.00 usr 28.74 sys + 0.00 cusr 0.00 csys = 28.74 CPU) Time for select_key2_return_key (200000): 63 wallclock secs ( 0.00 usr 28.81 sys + 0.00 cusr 0.00 csys = 28.81 CPU) Time for select_key2_return_prim (200000): 64 wallclock secs ( 0.00 usr 29.03 sys + 0.00 cusr 0.00 csys = 29.03 CPU) Test of compares with simple ranges Time for select_range_prefix (20000:43500): 2 wallclock secs ( 0.00 usr 1.70 sys + 0.00 cusr 0.00 csys = 1.70 CPU) Time for select_range_key2 (20000:43500): 2 wallclock secs ( 0.00 usr 1.58 sys + 0.00 cusr 0.00 csys = 1.58 CPU) Time for select_group (111): 2 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Time for min_max_on_key (15000): 4 wallclock secs ( 0.00 usr 2.07 sys + 0.00 cusr 0.00 csys = 2.07 CPU) Time for min_max (60): 8 wallclock secs ( 0.00 usr 0.01 sys + 0.00 cusr 0.00 csys = 0.01 CPU) Time for count_on_key (100): 11 wallclock secs ( 0.00 usr 0.03 sys + 0.00 cusr 0.00 csys = 0.03 CPU) Time for count (100): 22 wallclock secs ( 0.00 usr 0.03 sys + 0.00 cusr 0.00 csys = 0.03 CPU) Time for count_distinct_big (20): 3 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing update of keys with functions Time for update_of_key (50000): 67 wallclock secs ( 0.00 usr 2.73 sys + 0.00 cusr 0.00 csys = 2.73 CPU) Time for update_of_key_big (501): 36 wallclock secs ( 0.00 usr 0.04 sys + 0.00 cusr 0.00 csys = 0.04 CPU) Testing update with key Time for update_with_key (300000): 122 wallclock secs ( 0.00 usr 15.61 sys + 0.00 cusr 0.00 csys = 15.61 CPU) Time for update_with_key_prefix (100000): 44 wallclock secs ( 0.00 usr 7.12 sys + 0.00 cusr 0.00 csys = 7.12 CPU) Testing update of all rows Time for update_big (10): 31 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing left outer join Time for outer_join_on_key (10:10): 1 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for outer_join (10:10): 2 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for outer_join_found (10:10): 2 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for outer_join_not_found (500:10): 2 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing SELECT ... WHERE id in (10 values) Time for select_in (500:5000) 1 wallclock secs ( 0.00 usr 0.10 sys + 0.00 cusr 0.00 csys = 0.10 CPU) Time for select_join_in (500:5000) 0 wallclock secs ( 0.00 usr 0.08 sys + 0.00 cusr 0.00 csys = 0.08 CPU) Testing SELECT ... WHERE id in (100 values) Time for select_in (500:50000) 0 wallclock secs ( 0.00 usr 0.29 sys + 0.00 cusr 0.00 csys = 0.29 CPU) Time for select_join_in (500:50000) 0 wallclock secs ( 0.00 usr 0.29 sys + 0.00 cusr 0.00 csys = 0.29 CPU) Testing SELECT ... WHERE id in (1000 values) Time for select_in (500:500000) 2 wallclock secs ( 0.00 usr 1.91 sys + 0.00 cusr 0.00 csys = 1.91 CPU) Time for select_join_in (500:500000) 1 wallclock secs ( 0.00 usr 1.82 sys + 0.00 cusr 0.00 csys = 1.82 CPU) Testing INSERT INTO ... SELECT Time for insert_select_1_key (1): 4 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for insert_select_2_keys (1): 4 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Time for drop table(2): 0 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Testing delete Time for delete_key (10000): 5 wallclock secs ( 0.00 usr 0.53 sys + 0.00 cusr 0.00 csys = 0.53 CPU) Time for delete_range (12): 6 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Insert into table with 16 keys and with a primary key with 16 parts Time for insert_key (100000): 1417 wallclock secs ( 0.00 usr 7.89 sys + 0.00 cusr 0.00 csys = 7.89 CPU) Testing update of keys Time for update_of_primary_key_many_keys (256): 6737 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Deleting rows from the table Time for delete_big_many_keys (128): 4525 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Deleting everything from table Time for delete_all_many_keys (1): 4525 wallclock secs ( 0.00 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.02 CPU) Inserting 100000 rows with multiple values Time for multiple_value_insert (100000): 3 wallclock secs ( 0.00 usr 0.20 sys + 0.00 cusr 0.00 csys = 0.20 CPU) Time for drop table(1): 0 wallclock secs ( 0.00 usr 0.00 sys + 0.00 cusr 0.00 csys = 0.00 CPU) Total time: 13765 wallclock secs (57.02 usr 279.48 sys + 0.00 cusr 0.00 csys = 336.50 CPU)
その間時間かかるけどscreenでこんなコマンド打ってから眺めると良いです。
% vmstat 5
% iostat -dx 5
topコマンドはShift+Mでメモリ順にソートできます。
top
ときどきCPU使用率が100%超えたりスワップしていたりするところが見えるので面白いと思います。
