Median runtime (wall-clock) in 3 runs using 4 threads, and number of successful runs (*-***) Best sequential solver with 3600s time-out Speed-up (parallel vs. sequential) Relative variation coefficient (100∙σ/µ/Ăn %) for up to three (n) measurements
Solver      
SAT/ ManySat   pMiniSat MiraXT MiniSat ManySat pMiniSat MiraXT ManySat pMiniSat MiraXT
Instance UNSAT (32bit)   (64bit) (32bit) (32bit) (32bit) (64bit) (32bit) (32bit) (64bit) (32bit)
IBM Benchmark Suite (15 SAT / 5 UNSAT)
ibm-2002-04r-k80 SAT 20 *** 19 *** 12 *** 17,4 4,79 0,86 2,30 12% 7% 16%
ibm-2002-11r1-k45 SAT 18 *** 13 *** 22 *** 15,5 3,18 1,54 0,49 8% 4% 27%
ibm-2002-18r-k90 SAT 139 *** 68 *** 243,5 * 81,9 2,20 0,90 3,53 16% 12% 7%
ibm-2002-20r-k75 SAT 112 *** 59 *** 120 *** 50,3 4,26 1,73 3,75 12% 24% 33%
ibm-2002-22r-k60 UNSAT 129 *** 265 *** -----   202,5 3,94 1,29 ----- 4% 2% -----
ibm-2002-22r-k75 SAT 82 *** 40 *** 78 *** 46,0 3,00 2,09 2,96 22% 10% 22%
ibm-2002-22r-k80 SAT 151 *** 43 *** 86 *** 95,0 3,18 3,46 2,92 19% 22% 37%
ibm-2002-23r-k90 SAT 473 *** 123 *** 472 *** 141,5 (>1.90) 1,16 1,23 16% 6% 2%
ibm-2002-24r3-k100 UNSAT 43 *** 99 *** 759 *** 125,5 4,73 2,31 (>1.19) 13% 4% 11%
ibm-2002-25r-k10 UNSAT 582 *** 242 *** -----   669,5 (>1.55) 1,65 ----- 2% 4% -----
ibm-2002-29r-k75 SAT 124 *** 39 *** 228 *** 35,4 5,04 3,97 0,23 20% 31% 29%
ibm-2002-30r-k85 SAT -----   284 *** -----   1458,7 ----- 3,17 ----- ----- 37% -----
ibm-2002-31_1r3-k30 UNSAT 138 *** 176 *** -----   143,2 3,38 2,29 ----- 3% 3% -----
ibm-2004-01-k90 SAT 40 *** 15 *** 21 *** 17,4 1,75 0,85 5,59 26% 26% 24%
ibm-2004-1_11-k80 SAT 138 *** 61 *** 143 *** 53,0 4,09 0,63 0,54 10% 20% 18%
ibm-2004-23-k100 SAT 543 *** 119 *** 149 *** 107,8 (>1.66) 1,80 2,40 15% 9% 32%
ibm-2004-23-k80 SAT 180 *** 68 *** 57 *** 260,7 (>5.00) 2,11 1,51 9% 8% 14%
ibm-2004-29-k25 UNSAT 51 *** 34 *** 305 *** 42,2 3,71 2,02 2,21 7% 5% 17%
ibm-2004-29-k55 SAT 44 *** 19 *** 21 *** 26,5 1,77 0,55 0,65 17% 27% 22%
ibm-2004-3_02_3-k95 SAT 2 *** 2 *** 1 *** 1,5 2,79 0,81 1,69 12% 0% 20%
Manolios Benchmark Suite (10 UNSAT)
manol-pipe-c10nid_i UNSAT 205 *** -----   -----   384,9 4,03 (<0.46) ----- 2% ----- -----
manol-pipe-c10nidw UNSAT 408 *** -----   -----   535,1 (>2.21) ----- ----- 3% ----- -----
manol-pipe-c6bidw_i UNSAT 21 *** 39 *** 57 *** 34,5 3,50 1,13 2,20 4% 5% 7%
manol-pipe-c8nidw UNSAT 128 *** 262 *** -----   204,1 4,08 0,79 (<0.90) 3% 3% -----
manol-pipe-c9n_i UNSAT 8 *** 8 *** 80 *** 13,3 4,31 2,12 1,64 0% 4% 15%
manol-pipe-f7nidw UNSAT 41 *** 259 *** 76 *** 494,3 3,63 (>3.47) 2,68 3% 19% 12%
manol-pipe-f9b UNSAT 96 *** 239 *** 486 *** 356,1 3,25 2,41 (>1.85) 6% 17% 10%
manol-pipe-g10bid_i UNSAT 202 *** -----   -----   393,1 4,12 (<0.61) ----- 3% ----- -----
manol-pipe-g10nid UNSAT 40 *** 69 *** 139 *** 152,2 3,78 2,11 3,28 6% 5% 17%
manol-pipe-g8nidw UNSAT 10 *** 35 *** 22 *** 206,4 2,45 10,57 1,31 5% 13% 2%
Mironov-Zhang Benchmark Suite (10 SAT)
mizh-md5-47-3 SAT 492 *** 86 *** 21 *** 20,0 (>1.83) 2,78 5,91 30% 22% 28%
mizh-md5-47-4 SAT 144 *** 50 *** 226 *** 40,1 (>6.25) 7,16 1,29 45% 52% 31%
mizh-md5-47-5 SAT 445 ** 45 *** 113 *** 475,4 1,39 2,23 0,24 ----- 33% 31%
mizh-md5-48-2 SAT 398 * 128 *** 113 *** 124,5 (>2.26) 2,10 3,83 40% 1% 35%
mizh-md5-48-5 SAT 440,5 * 286 *** 335 *** 93,7 (>2.04) 2,45 1,16 56% 43% 29%
mizh-sha0-35-3 SAT 39 *** 8 *** 30 *** 7,5 (>23.08) 4,51 0,67 47% 60% 43%
mizh-sha0-35-4 SAT 258,5 * 31 *** 54 *** 9,5 0,40 6,71 1,82 17% 21% 30%
mizh-sha0-36-1 SAT 244 ** 117 *** 371 * 189,9 (>3.69) (>7.69) (>2.43) ----- 53% 7%
mizh-sha0-36-3 SAT 286,5 * 54 *** 674 *** 68,1 (>3.14) 6,24 (>1.34) 31% 58% 12%
mizh-sha0-36-4 SAT 189,5 * 84 *** 729 ** 367,4 (>4.75) (>10.71) (>1.23) 12% 19% -----
Post Benchmark Suite (2 SAT, 8 UNSAT)
post-c32s-col400-16 UNSAT 90 *** 20 *** 277 *** 47,0 2,99 1,76 0,86 3% 8% 24%
post-c32s-gcdm16-22 SAT 88 *** 122 *** -----   103,1 4,11 1,21 ----- 4% 14% -----
post-c32s-gcdm16-23 UNSAT 88 *** 145 *** -----   125,8 3,23 1,13 ----- 4% 9% -----
post-c32s-ss-8 UNSAT 415 *** 425 *** -----   1028,1 (>2.17) 0,80 ----- 1% 3% -----
post-cbmc-aes-d-r1 UNSAT 3 *** 6 *** 4 *** 2,8 1,53 0,83 1,06 0% 5% 7%
post-cbmc-aes-d-r2 UNSAT 225 *** 190 *** -----   1037,3 3,61 3,58 ----- 1% 2% -----
post-cbmc-aes-ee-r2 UNSAT 236 *** 179 *** -----   1135,0 3,10 4,47 ----- 10% 2% -----
post-cbmc-aes-ee-r3 UNSAT 872 ** -----   -----   3241,1 (>1.03) ----- ----- ----- ----- -----
post-cbmc-aes-ele UNSAT 33 *** 41 *** 34 *** 20,2 1,45 1,08 2,49 1% 1% 5%
post-cbmc-zfcp-2.8-u2 SAT 118 *** 34 *** -----   88,5 (>7.63) (>26.47) (<0.04) 4% 0% -----
Velev Benchmark Suite (4 SAT, 6 UNSAT)
velev-engi-uns-1.0-4nd UNSAT 9 *** 4 *** 7 *** 9,0 4,58 2,69 4,26 5% 0% 0%
velev-fvp-sat-3.0-b18 SAT 18 *** 21 *** 9 *** 18,6 0,40 9,20 3,34 12% 26% 34%
velev-npe-1.0-9dlx-b71 SAT 651 *** 483,5 * 168 *** 106,1 (>1.38) 0,35 3,90 8% 7% 31%
velev-vliw-sat-4.0-b4 SAT 55 *** 27 *** 33 *** 49,1 1,76 0,79 0,73 24% 4% 8%
velev-vliw-sat-4.0-b8 SAT 56 *** 32 *** 38 *** 72,5 1,50 1,04 0,91 8% 3% 4%
velev-vliw-uns-2.0-iq1 UNSAT 163 *** 330 *** -----   ----- 4,18 (>2.73) ----- 11% 5% -----
velev-vliw-uns-2.0-iq2 UNSAT 601 *** -----   -----   ----- (>1.50) ----- ----- 5% ----- -----
velev-vliw-uns-2.0-uq5 UNSAT -----   -----   -----   ----- ----- ----- ----- ----- ----- -----
velev-vliw-uns-4.0-9-i1 UNSAT -----   -----   -----   ----- ----- ----- ----- ----- ----- -----
velev-vliw-uns-4.0-9 UNSAT 787 *** -----   -----   ----- (>1.14) ----- ----- 4% ----- -----
Mixed Benchmark Suite (17 SAT, 23 UNSAT)
aloul-chnl11-13 UNSAT -----   -----   696 *** ----- ----- ----- (>1.29) ----- ----- 4%
anbul-dated-5-15-u UNSAT 200 *** 85 *** 886 * 375,8 4,09 1,67 (>1.02) 18% 11% -----
anbul-part-10-13-s SAT 898 ** -----   -----   ----- (>1.00) ----- ----- ----- ----- -----
anbul-part-10-15-s SAT 786 * -----   493 ** 134,5 (>1.15) ----- (>1.83) 3% ----- -----
babic-dspam-vc1080 UNSAT 2 *** 2 *** 2 *** 1,2 0,86 0,69 0,74 0% 0% 16%
babic-dspam-vc949 UNSAT 2 *** 1 *** 1 *** 1,0 0,76 1,23 1,09 0% 0% 0%
babic-dspam-vc973 UNSAT 4 *** 3 *** 3 *** 2,6 0,86 0,85 0,90 0% 10% 0%
cmu-bmc-barrel6 UNSAT 1 *** 1 *** 3 *** 1,8 3,01 1,86 1,67 20% 0% 10%
cmu-bmc-longmult13 UNSAT 8 *** 4 *** 6 *** 27,9 4,37 3,13 3,87 14% 6% 13%
cmu-bmc-longmult15 UNSAT 10 *** 4 *** 4 *** 16,1 4,74 2,97 3,65 3% 0% 12%
een-pico-prop00-75 UNSAT 2 *** 2 *** 2 *** 2,0 1,33 1,06 0,77 12% 0% 0%
een-pico-prop05-75 UNSAT 11 *** 14 *** 59 *** 14,9 3,49 1,30 0,86 4% 7% 13%
een-tip-sat-nusmv-t5.B SAT 1 *** 1 *** 1 *** 0,9 2,49 0,99 1,15 0% 20% 0%
een-tip-sat-texas-tp-5e SAT 1 *** 0,5 *** 0,5 *** 0,1 0,25 0,34 0,26 41% 54% 0%
een-tip-sat-vis-eisen SAT 1 *** 0,5 *** 0,5 *** 0,3 0,60 0,68 0,88 41% 54% 54%
fuhs-aprove-15 UNSAT 13 *** 8 *** 67 *** 15,2 2,89 3,44 1,12 9% 9% 8%
fuhs-aprove-16 UNSAT 198 *** 86 *** 308 *** 162,5 3,65 2,23 (>2.92) 4% 11% 4%
goldb-heqc-alu4mul UNSAT 143 *** 41 *** 107 *** 192,8 4,46 4,03 3,09 7% 6% 17%
goldb-heqc-dalumul UNSAT -----   208 *** -----   1713,5 ----- 3,92 ----- ----- 2% -----
goldb-heqc-frg1mul UNSAT -----   -----   -----   ----- ----- ----- ----- ----- ----- -----
goldb-heqc-x1mul UNSAT -----   -----   -----   ----- ----- ----- ----- ----- ----- -----
grieu-vmpc-27 SAT 1 *** 11 *** 83 *** 129,0 250,17 1,80 0,67 59% 53% 6%
grieu-vmpc-31 SAT 96,5 * -----   432 ** 1608,2 0,40 ----- (>2.08) 63% ----- -----
hoons-vbmc-lucky7 UNSAT 1 *** 1 *** 14 *** 1,1 3,47 1,42 1,61 20% 0% 29%
jarvi-eq-atree-9 UNSAT 55 *** 18 *** 31 *** 150,0 4,37 3,35 2,10 4% 3% 10%
marijn-philips UNSAT -----   460 *** 466 *** 2506,4 ----- (>1.96) 1,61 ----- 9% 1%
maris-s03-gripper11 SAT 15 *** 107 *** -----   1025,0 1,59 0,62 ----- 28% 52% -----
narain-vpn-clauses-10 SAT 106 *** 36 *** 141 *** 75,0 3,45 (>25.00) 1,23 10% 3% 4%
narain-vpn-clauses-8 SAT -----   49 *** 729 ** 831,3 ----- (>18.37) (>1.23) ----- 8% -----
palac-sn7-ipc5-h16 SAT 66 *** 96 *** 166 *** 283,7 2,36 3,45 1,92 11% 26% 12%
palac-uts-l06-ipc5-h34 SAT 19 *** 29 *** 71 *** 18,0 1,66 1,20 0,58 5% 3% 8%
schup-l2s-abp4-1-k31 UNSAT 8 *** 8 *** 59 *** 8,3 4,49 1,62 0,77 9% 14% 10%
schup-l2s-bc56s-1-k391 UNSAT 440 *** 720 *** -----   515,5 (>2.05) 1,00 ----- 2% 14% -----
schup-l2s-motst-2-k315 SAT 28 *** 46 *** 53 *** 23,9 3,94 0,58 0,86 12% 15% 2%
simon-s02-f2clk-50 UNSAT 102 *** 103 *** -----   172,8 4,02 2,14 ----- 3% 8% -----
simon-s02b-r4b1k1.1 SAT 282 *** 124 *** 248 *** 146,0 0,93 1,86 1,10 39% 13% 4%
simon-s02b-r4b1k1.2 SAT 58 *** 235 *** 119 *** 105,7 14,06 1,11 (>7.56) 51% 35% 5%
simon-s03-fifo8-400 UNSAT 18 *** 19 *** 78 *** 35,2 3,23 1,59 1,15 4% 6% 2%
simon-s03-w08-15 SAT 52 *** 31 *** 113 *** 73,9 3,59 2,68 0,45 16% 16% 31%
vange-col-abb313GPIA-9-c SAT -----   -----   31 *** ----- ----- ----- 1,01 ----- ----- 8%
#solved: 90 85 73 90 average speed-up: 6,02 3,10 1,83 runtime variation: 13,7% 14,7% 15,2%
#solved in all three runs: 79 84 66 by SAT/UNSAT: 8.84 / 3.14 4.00 / 2.18 1.85 / 1.81 by SAT/UNSAT: 22.2%/5.5% 23.1%/5.7% 19.5%/9.7%
#solved in ˛900s CPU time: 69 72 53 81 mininal speed-up: 0,25 0,34 0,04
#solved SAT/UNSAT: 45/45 44/41 43/30 46/44 by SAT/UNSAT: 0.25 / 0.76 0.34 / 0.46 0.04 / 0.74
maximal speed-up: 250,17 26,47 7,56
rank: 1   2   3   by SAT/UNSAT: 250.17 / 4.74 26.47 / 10.57 7.56 / 4.26