Contact the author.
This document contains results of some performance tests run with various configurations. Not much analysis here, just some data. Contributed by Brian, January 2003. ---------------------------------- I (Brian) have done a few tests on the storage devices that I have tried out. If you have tested OpenGFS on something else or you have different numbers, let me know. I will run more tests as time allows. Here's a link to a Cisco webpage about Fibre Channel versus iSCSI perfomance: http://www.cisco.com/warp/public/cc/pd/rt/5420/prodlit/csubl_br.htm Media Max Read ----- -------- ieee1394 12MB/s iSCSI 9MB/s The ieee1394 setup was an old WD310100 10GB UDMA33 in an ADS Pyro firewire drive box (uses the oxford ieee1394->ide bridge chip) connected between a P2 450 and a Dual P2 450 with ieee1394 (Duh!?!) The iSCSI setup was an Athlon XP 1800+ with an ide raid setup with 2xWD800JB acting as the target, and 2 K6-2 266's as the "clients" on fast ethernet (not switched). Contributed by Brian, April-June 2003. ---------------------------------- Results from running OpenGFS on OSDL machines. ---2.4.19-1 drive-1 subpool Version 1.02c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP cl038 496M 6209 62 10723 13 6471 8 8678 82 18728 9 210.7 3 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 479 19 1811 25 323 4 1126 26 +++++ +++ 233 4 cl038,496M,6209,62,10723,13,6471,8,8678,82,18728,9,210.7,3,16,479,19,1811,25,323,4,1126,26,+++++,+++,233,4 ---2.4.20-8 drives-8 subpools cl049,5000M,6000,59,10612,12,7364,14,9827,92,23422,13,148.2,1,16,439,20,+++++,+++,257,6,897,27,+++++,+++,247,4 ---2.4.20-8 drives-8 subpools-noatime cl049,5000M,5950,59,10501,12,7373,14,9858,93,21995,11,111.2,2,16,105,4,2556,27,274,4,1010,30,+++++,+++,229,5 cl049,5000M,5956,59,10490,12,7358,14,9866,93,22017,11,111.0,2,16,100,4,2635,31,276,5,896,27,+++++,+++,215,4 ---2.4.20-8 drives-8 subpools-noatime-w/shared_writ_mmap cl049,5000M,5984,59,10467,12,7393,14,9905,93,21962,11,110.3,2,16,100,4,2992,26,275,6,940,29,2208,25,213,4 2003/05/31 The below tests were all run on a single node setup all on the same exact drive just mounted with different diralloc methods, without restarting inbetween. Machine only has 512MB ram so the tests couldn't be totally cached DAMETH_RANDOM spyder,1G,6087,65,14476,12,8277,6,9602,92,18033,5,175.4,1,16,776,21,+++++,+++,113,1,593,14,9486,63,115,1 spyder,1G,6069,65,14500,12,8015,6,9531,91,16600,4,175.7,1,16,848,21,18115,70,112,1,567,14,+++++,+++,112,1 spyder,1G,6081,65,14286,11,7999,6,9535,91,16581,5,176.1,1,16,72,1,8858,70,117,1,680,18,+++++,+++,115,1 DAMETH_SINGLE spyder,1G,6092,65,14027,11,7909,5,9475,90,16177,5,171.6,0,16,829,22,+++++,+++,111,1,629 ,17,18969,74 ,112,1 spyder,1G,5773,62,13947,11,7856,6,9468,90,16240,4,158.0,0,16,801,21,12045,65 ,113,1,570 ,16,6450 ,66 ,113,1 spyder,1G,5747,62,13939,11,7863,6,9456,90,16247,4,160.2,1,16,825,22,+++++,+++,111,1,585 ,15,+++++,+++,111,1 spyder,1G,6031,64,13410,10,7769,5,9411,90,16009,5,137.0,0,16,73 ,1 ,3127 ,31 ,119,1,1670,28,+++++,+++,77 ,1 spyder,1G,5799,62,13346,10,7782,6,9457,90,16035,4,136.1,1,16,72 ,1 ,6992 ,68 ,117,1,1733,31,+++++,+++,75 ,1 spyder,1G,5795,62,13471,11,7810,6,9469,90,16039,4,138.4,0,16,72 ,1 ,2417 ,24 ,123,1,1717,29,+++++,+++,75 ,1 spyder,1G,5797,62,13421,10,7741,5,9382,90,16031,5,136.1,0,16,71 ,1 ,7196 ,65 ,122,1,1661,30,+++++,+++,76 ,1 DAMETH_RROBIN spyder,1G,6064,65,14048,11,7869,6,9456,90,16277,5,157.9,0,16,738,19,+++++,+++,113,1,613,16,27336,70 ,113,1 spyder,1G,5785,62,13866,11,7845,5,9457,90,16278,4,150.1,0,16,70 ,1 ,7442 ,70 ,121,1,593,17,+++++,+++,115,1 spyder,1G,5666,61,13561,10,7857,6,9479,90,16311,4,153.1,0,16,850,22,+++++,+++,107,1,638,18,8755 ,66 ,112,1 spyder,1G,6058,65,13296,11,7776,5,9429,90,15923,4,148.8,0,16,804,21,+++++,+++,112,1,649,17,+++++,+++,114,1 spyder,1G,5668,61,13410,11,7753,5,9412,90,15965,4,149.5,1,16,802,20,+++++,+++,114,1,624,17,25736,67 ,113,1 spyder,1G,5865,63,13278,10,7799,6,9396,90,15923,5,140.5,0,16,796,20,+++++,+++,113,1,575,15,15162,70 ,113,1 spyder,1G,5678,61,13228,10,7741,5,9440,90,15957,4,140.8,0,16,72 ,1 ,7735 ,67 ,121,1,594,15,21861,70 ,114,1 avg-rr,1G,5826,63,13527,11,7806,5,9438,90,16091,4,148.7,0,16,798,20,7589 ,69 ,114,1,612,16,24978,69 ,113,1 IOzone [root@cl049 ogfs]# /tmp/osdl.project/src/current/iozone -a Iozone: Performance Test of File I/O Version $Revision: 1.2 $ Compiled for 32 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Jean-Marc Zucconi, Jeff Blomberg. Run began: Sun Dec 1 21:34:33 2002 Auto Mode Command line used: /tmp/osdl.project/src/current/iozone -a Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 64 4 70639 7300 438279 853341 576300 7235 405107 118520 609573 6737 7266 363648 711267 64 8 103904 5955 571343 984256 752479 6836 484972 167109 695243 6347 6652 453818 864412 64 16 130337 5119 516183 954871 790194 7036 537908 203787 615714 5818 6234 453961 780235 64 32 135023 4653 378811 955665 852868 6400 496249 185515 546785 5462 5839 420897 711217 64 64 133322 5183 503988 842400 646591 5129 457156 132216 361484 4129 4175 447404 727384 128 4 75250 12967 412876 595289 547043 11525 376475 133478 324896 12160 11339 358582 561369 128 8 111114 13911 428142 684517 646625 13100 419707 203506 785068 11125 9291 403726 652994 128 16 143491 11234 486738 753140 761822 13448 481250 270616 757304 11029 10892 422474 646477 128 32 155349 12498 494204 743934 731262 12380 494105 280693 723401 7815 9904 391490 735579 128 64 145948 10208 439928 612545 711139 10243 438304 201592 533439 7823 8263 347793 540102 128 128 151308 7834 343134 406317 357521 7744 278252 144793 315285 6237 6271 337683 379872 256 4 72644 23961 375892 486741 490441 24353 8054 144468 326101 20940 19595 341765 410240 256 8 105918 22387 410256 515150 568945 24471 381553 218614 337740 20023 15518 383799 455514 256 16 124270 24151 448308 533358 599471 25147 442880 327375 820576 21485 18637 420399 430232 256 32 142302 22178 468019 574001 668308 23247 468016 363641 870801 19485 15512 429572 508977 256 64 137266 19420 361082 421076 436076 19929 367312 293593 526743 15014 15662 344993 403793 256 128 138980 15085 325685 353600 361054 15056 322018 193796 360578 12450 11805 309918 337738 256 256 129226 9142 240613 242886 234644 9228 247112 137188 248051 9959 10411 231471 243115 512 4 68193 36165 362608 385538 400921 39251 340205 149051 329259 17829 8198 336409 355560 512 8 97672 41819 401253 423121 480737 43656 389656 241281 375908 33143 31395 379538 405371 512 16 119318 37042 440610 449509 515628 44591 428469 363643 427744 35811 33909 422106 461240 512 32 127999 41454 452707 492775 548181 42384 455132 439846 949956 33236 31439 434261 477166 512 64 128225 34527 380967 398758 483476 36988 396902 338403 858995 29882 33339 341777 394758 512 128 132744 25194 306592 310491 291399 29904 307329 221257 363908 24982 24813 289105 300123 512 256 137043 22093 236054 238250 235620 21739 241508 164311 241169 19538 20068 237035 233255 512 512 124452 13986 209917 212890 211220 13366 211831 134208 212007 12114 12420 212979 213596 1024 4 66614 54159 356176 387872 376752 55205 332679 150587 337842 42307 33641 339297 359924 1024 8 94535 62526 394304 409768 438919 64333 391731 249150 386275 47577 27823 385100 406662 1024 16 112862 66030 439115 470586 487630 49566 439665 385685 423132 53358 53693 423850 457144 1024 32 119166 63784 460443 489728 485536 56841 465661 512011 464389 50448 39510 431162 480087 1024 64 119920 57322 386421 403776 406020 41960 378283 455113 577870 47313 44564 368607 390534 1024 128 125814 50000 289187 301802 304403 49247 303681 264732 394300 36089 14826 271835 295443 1024 256 130979 32583 227204 234699 230889 32308 231623 181913 240655 26246 26461 223728 232619 1024 512 134188 23941 194787 201101 200429 26744 198642 154286 200547 20670 21626 197187 198528 1024 1024 126638 15174 190440 192917 192589 14627 189558 129292 190829 13859 13798 186588 191366 2048 4 50755 55872 369338 378627 348242 62385 340256 151771 341052 43909 55945 345537 357164 2048 8 84586 76615 396671 424634 413735 26010 395826 252901 393545 59083 76738 390843 406266 2048 16 99191 88997 447943 457250 467047 91277 435470 401808 440336 58218 81818 426671 456937 2048 32 103980 89074 465456 476501 492423 90519 458681 553510 468976 64202 24399 458269 465879 2048 64 109156 79636 373519 385108 392638 81736 378838 596210 372903 62215 68073 372094 381876 2048 128 113771 72464 299371 304040 306815 73473 296766 283265 411907 45099 60737 294086 300648 2048 256 117816 61210 231359 233524 232042 49524 231517 194104 232330 47584 51383 228750 233336 2048 512 78772 46895 192753 195233 193408 43736 194676 165467 193408 35245 40644 190387 193793 2048 1024 116735 28708 182385 184437 183676 29993 184057 149816 182709 28829 30050 181287 184107 2048 2048 114798 16989 182093 183908 183513 16208 183874 108486 182923 13199 14949 181641 183331 4096 4 61502 57703 367916 384675 347294 76312 344026 152545 339580 53164 63481 346062 359961 4096 8 84526 97889 406548 422877 403865 97077 395750 253999 394037 68222 72926 391884 407320 4096 16 101153 109806 446136 466462 457091 109912 445268 404344 445263 78093 80723 434034 457809 4096 32 104746 101905 464135 487850 485826 110565 471180 559867 469560 80430 80506 458060 482222 4096 64 107182 102185 379295 392824 388137 104007 384350 425469 383950 72946 73170 187924 387475 4096 128 111686 91220 244479 255297 243446 63230 261408 245607 269244 60599 61741 291882 298693 4096 256 101905 87355 229635 233110 232251 77743 231988 196714 236721 56881 48168 227138 231988 4096 512 119222 71910 189454 191285 190564 68661 190352 170702 191840 53873 46714 188287 190715 4096 1024 120772 31664 178521 180544 180297 49016 180266 158575 180250 43357 37746 178280 179916 4096 2048 122447 31297 178040 161935 86455 24351 180051 125217 179224 30867 25484 176918 178981 4096 4096 121962 17387 178140 180162 179719 16948 179846 120818 179854 15156 14063 177808 179917 8192 4 60654 71288 373877 388873 345843 77357 350040 153150 342775 52592 67428 350325 364364 8192 8 78414 82122 412258 426599 112113 105630 400979 257278 397941 73330 84482 397264 412841 8192 16 98270 101691 450752 470778 453071 121183 449787 409089 449516 83249 100133 440856 460482 8192 32 100706 98423 467953 489628 482620 125246 474650 572789 475092 87855 87950 462693 141387 8192 64 105371 95532 380316 394301 389336 120267 386452 369207 387897 81793 84805 378383 391026 8192 128 65573 116312 295697 302243 298684 113058 299942 300711 300459 73590 43490 292571 300271 8192 256 114912 84730 229551 232767 231857 108216 232701 202511 232219 70578 65147 228030 231020 8192 512 118698 79801 188534 190578 189520 96975 190175 174613 190458 67917 63334 187928 190436 8192 1024 121381 62061 176612 178502 177820 54702 177986 163418 177743 57339 53285 176024 177939 8192 2048 121235 42225 175718 177343 176925 51101 177323 155127 178017 41571 38996 175173 177089 8192 4096 123173 27414 175331 176933 176700 31192 176605 143266 167600 27391 27551 175507 177101 8192 8192 123330 16064 139840 177481 177020 17390 177423 118134 177101 13619 17199 175526 177335 16384 4 58606 80935 377354 388348 343537 83842 351753 154295 344788 41948 79999 353675 364348 16384 8 80118 99275 412737 426933 399249 110881 401479 257537 397834 74343 94421 399609 413559 16384 16 97551 113164 453159 468435 450976 127035 452359 411502 450023 84913 115626 443613 459180 16384 32 105184 123122 470466 489878 480624 130424 477055 586022 477278 58422 81635 465785 483403 16384 64 102065 108354 386971 399055 396725 127010 391064 456964 396016 85330 97816 385406 397419 16384 128 110607 73110 297290 302678 300590 125438 301132 280600 301248 76568 89967 294968 301481 16384 256 111983 118746 229243 231746 230666 124189 232097 203361 139065 74116 81592 228056 231320 16384 512 89690 102994 187425 189408 189392 105850 189563 175613 189152 73818 76347 186901 189469 16384 1024 116262 85108 175694 177244 177110 95558 176770 165337 177812 34607 69596 175108 176988 16384 2048 121589 72747 174602 176331 176092 74320 176696 162017 177112 51320 56940 174578 176065 16384 4096 123581 45067 174835 176490 176675 45971 176748 155003 176715 27245 40806 174917 176485 16384 8192 109915 29375 174615 176257 176001 33155 176088 127774 176299 31819 30732 175010 176380 16384 16384 123021 17000 174919 176599 176557 18532 176160 119753 176557 13776 18239 174515 175914 32768 64 98618 130164 386137 396981 394382 83139 393704 404054 392600 28940 105264 382138 392930 32768 128 27759 128980 297226 303098 300800 133460 300861 311426 300905 36502 96051 227794 300783 32768 256 33089 124091 229146 232423 231317 124285 231584 207993 231456 42238 93240 227937 231187 32768 512 30152 120931 187183 189110 189097 114012 189009 144011 188953 64096 85713 186847 188981 32768 1024 31469 116976 174430 176897 176349 110822 176760 166916 176590 75837 83294 175085 176807 32768 2048 36482 93870 174485 176010 175874 96576 175818 66270 175556 43103 72725 173771 175349 32768 4096 39641 71409 174367 144549 175805 69214 175908 162294 176080 47885 59084 174172 175818 32768 8192 46661 48011 148323 175683 175562 53008 175755 115698 175619 21044 38553 174232 175925 32768 16384 55833 32164 174268 176016 150172 30515 176158 140927 175977 28744 23955 173642 175900 65536 64 68086 136417 388023 398824 393820 138480 394149 339043 394388 75280 113639 384842 395145 65536 128 41283 137695 297999 304148 301091 139943 301847 281009 301631 56044 100601 295047 300614 65536 256 45937 137421 228811 231827 231041 141224 231743 204874 231616 56649 96817 228076 231384 65536 512 54178 120339 157413 188901 188539 141603 188759 176545 147982 80005 95478 186989 188685 65536 1024 51149 130750 174951 176618 176578 135816 176306 168764 176369 79922 93992 174742 176272 65536 2048 46902 116600 173837 175550 175321 107414 175446 79300 175663 69118 88330 161549 175707 65536 4096 53272 97768 174096 175553 175680 103211 175529 55197 175639 71456 77574 173983 175611 65536 8192 66119 68829 174012 175477 149423 85412 175465 58839 162234 37890 61443 173882 175726 65536 16384 82632 53202 174041 175646 175725 55085 175622 87228 151509 25471 42154 174083 175738 131072 64 95429 138969 388988 399485 395999 132944 395552 363733 394739 78121 116849 387751 397869 131072 128 66718 137134 283145 303978 300613 85944 301870 263998 301664 82523 102859 295774 301179 131072 256 65918 144249 229386 232428 220037 143171 231438 202640 231023 77083 95431 219368 231200 131072 512 98849 144080 187014 188929 188818 148790 182807 177080 188796 79701 98147 181279 187869 131072 1024 96076 141096 174883 176474 176315 147678 176456 166924 176436 79362 98061 174789 176356 131072 2048 81811 127134 168863 169202 175559 141299 175588 127555 175573 82772 94488 173811 175467 131072 4096 107518 114333 173922 164209 175497 130260 175517 125186 175413 79942 80393 173877 175431 131072 8192 111901 98853 167655 170455 175559 113333 169235 156974 175443 48530 75527 167784 175518 131072 16384 107110 61933 174101 175687 175729 86208 175608 152998 175680 41663 60572 173808 165628 262144 64 13020 112700 42747 394455 393223 124052 389338 470243 393477 19506 113815 385381 394901 262144 128 15371 133769 298147 301697 300951 143124 298755 308377 301854 19473 58529 295764 298555 262144 256 20189 74484 227287 232317 229878 148490 230370 204816 231653 20036 60497 228493 230148 262144 512 20630 76719 185969 188965 187726 146623 188940 176540 186789 20298 60244 185737 188629 262144 1024 19955 74451 174773 176450 175275 132601 175237 168741 176344 20045 61698 174600 175410 262144 2048 20964 72563 173925 174659 174180 131313 175481 166014 175328 19150 67785 173985 175327 262144 4096 20765 66684 173942 175495 173607 142211 173601 166563 172731 18834 58338 173921 175298 262144 8192 20816 66239 174071 175670 173863 116763 174608 166340 174454 18056 77315 174024 175515 262144 16384 20418 99403 173984 175421 174730 104308 175450 161454 175540 17342 65884 173659 175516 524288 64 12963 15093 22037 24270 17824 132031 10041 581512 12881 11142 13853 20989 22032 524288 128 13595 19545 21568 23149 29501 133515 18420 291665 19196 11351 14196 20515 22792 524288 256 13370 33251 22248 22686 37552 134541 23054 204853 23615 11324 14258 20572 22033 524288 512 13405 33429 21823 22157 43202 140909 23402 176111 25303 11009 14075 19772 22400 524288 1024 13417 29700 21507 25063 47239 137215 26206 167976 27702 10962 14170 20494 22064 524288 2048 13250 27352 22660 22344 47140 135819 25911 167212 28402 11832 14004 19979 22673 524288 4096 13318 23560 20811 22956 49508 139333 28701 167150 29863 10867 14283 20522 21508 524288 8192 13518 23477 21198 22639 38236 135270 23791 166918 28509 11090 14007 19839 21725 524288 16384 13713 13689 19946 21716 43479 124958 26747 165479 29265 11328 13573 19962 21719 iozone test complete.