load module flavor/buildcompiler/gcc/12 load module binutils/2.42 load module c++/gcc/12.3.0 load module c/gcc/12.3.0 load module fortran/gcc/12.3.0 load module gcc/12.3.0 load module feature/nvhpc/cuda-12.4 load module c++/nvidia/24.5 load module c/nvidia/24.5 load module fortran/nvidia/24.5 load module nvhpc/24.5 HPCG-NVIDIA 24.3.0 -- NVIDIA accelerated HPCG benchmark -- NVIDIA Build v0.2.1 Start of application (GPU-Only) ... Initial Residual = 117499 Iteration = 1 Scaled Residual = 0.184375 Iteration = 2 Scaled Residual = 0.100949 Iteration = 3 Scaled Residual = 0.0696513 Iteration = 4 Scaled Residual = 0.0532241 Iteration = 5 Scaled Residual = 0.0430719 Iteration = 6 Scaled Residual = 0.036169 Iteration = 7 Scaled Residual = 0.0311748 Iteration = 8 Scaled Residual = 0.0274021 Iteration = 9 Scaled Residual = 0.0244585 Iteration = 10 Scaled Residual = 0.0220992 Iteration = 11 Scaled Residual = 0.0201626 Iteration = 12 Scaled Residual = 0.0185383 Iteration = 13 Scaled Residual = 0.0171497 Iteration = 14 Scaled Residual = 0.0159457 Iteration = 15 Scaled Residual = 0.0148927 Iteration = 16 Scaled Residual = 0.0139676 Iteration = 17 Scaled Residual = 0.0131538 Iteration = 18 Scaled Residual = 0.0124369 Iteration = 19 Scaled Residual = 0.0118023 Iteration = 20 Scaled Residual = 0.0112363 Iteration = 21 Scaled Residual = 0.0107258 Iteration = 22 Scaled Residual = 0.0102601 Iteration = 23 Scaled Residual = 0.00983229 Iteration = 24 Scaled Residual = 0.00943961 Iteration = 25 Scaled Residual = 0.0090814 Iteration = 26 Scaled Residual = 0.00875836 Iteration = 27 Scaled Residual = 0.00847035 Iteration = 28 Scaled Residual = 0.00821517 Iteration = 29 Scaled Residual = 0.00798872 Iteration = 30 Scaled Residual = 0.00778503 Iteration = 31 Scaled Residual = 0.00759774 Iteration = 32 Scaled Residual = 0.00742208 Iteration = 33 Scaled Residual = 0.0072548 Iteration = 34 Scaled Residual = 0.0070934 Iteration = 35 Scaled Residual = 0.0069348 Iteration = 36 Scaled Residual = 0.0067738 Iteration = 37 Scaled Residual = 0.00660458 Iteration = 38 Scaled Residual = 0.0064239 Iteration = 39 Scaled Residual = 0.00623341 Iteration = 40 Scaled Residual = 0.00604041 Iteration = 41 Scaled Residual = 0.00585507 Iteration = 42 Scaled Residual = 0.00568608 Iteration = 43 Scaled Residual = 0.00553845 Iteration = 44 Scaled Residual = 0.00541279 Iteration = 45 Scaled Residual = 0.0053058 Iteration = 46 Scaled Residual = 0.00521228 Iteration = 47 Scaled Residual = 0.00512699 Iteration = 48 Scaled Residual = 0.00504596 Iteration = 49 Scaled Residual = 0.00496714 Iteration = 50 Scaled Residual = 0.00488914 GPU Rank Info: | Storage Format = SELL, Optimization = HAND-TUNED | cuSPARSE version 12.3 | Reference CPU memory = 29040.19 MB | GPU Name: 'NVIDIA GH200 120GB' | GPU Memory Use: 62279 MB / 96768 MB | Process Grid: 8x8x29 | Local Domain: 512x512x288 | Number of CPU Threads: 72 WARNING: PERFORMING UNPRECONDITIONED ITERATIONS Call [0] Number of Iterations [11] Scaled Residual [1.19809e-14] WARNING: PERFORMING UNPRECONDITIONED ITERATIONS Call [1] Number of Iterations [11] Scaled Residual [1.19809e-14] Call [0] Number of Iterations [2] Scaled Residual [1.59492e-17] Call [1] Number of Iterations [2] Scaled Residual [1.59492e-17] Departure from symmetry (scaled) for SpMV abs(x'*A*y - y'*A*x) = 1.93174e-16 Departure from symmetry (scaled) for MG abs(x'*Minv*y - y'*Minv*x) = 0 SpMV call [0] Residual [0] SpMV call [1] Residual [0] Initial Residual = 117499 Iteration = 1 Scaled Residual = 0.221227 Iteration = 2 Scaled Residual = 0.119995 Iteration = 3 Scaled Residual = 0.0817059 Iteration = 4 Scaled Residual = 0.0619071 Iteration = 5 Scaled Residual = 0.0498836 Iteration = 6 Scaled Residual = 0.0417754 Iteration = 7 Scaled Residual = 0.0359518 Iteration = 8 Scaled Residual = 0.0315706 Iteration = 9 Scaled Residual = 0.0281456 Iteration = 10 Scaled Residual = 0.0253979 Iteration = 11 Scaled Residual = 0.0231451 Iteration = 12 Scaled Residual = 0.0212619 Iteration = 13 Scaled Residual = 0.0196684 Iteration = 14 Scaled Residual = 0.0183014 Iteration = 15 Scaled Residual = 0.0171147 Iteration = 16 Scaled Residual = 0.0160766 Iteration = 17 Scaled Residual = 0.0151615 Iteration = 18 Scaled Residual = 0.0143502 Iteration = 19 Scaled Residual = 0.013627 Iteration = 20 Scaled Residual = 0.0129791 Iteration = 21 Scaled Residual = 0.0123968 Iteration = 22 Scaled Residual = 0.0118719 Iteration = 23 Scaled Residual = 0.0113971 Iteration = 24 Scaled Residual = 0.0109654 Iteration = 25 Scaled Residual = 0.0105692 Iteration = 26 Scaled Residual = 0.0102027 Iteration = 27 Scaled Residual = 0.00985993 Iteration = 28 Scaled Residual = 0.00953495 Iteration = 29 Scaled Residual = 0.00922384 Iteration = 30 Scaled Residual = 0.00892363 Iteration = 31 Scaled Residual = 0.00863351 Iteration = 32 Scaled Residual = 0.0083549 Iteration = 33 Scaled Residual = 0.00808867 Iteration = 34 Scaled Residual = 0.00783667 Iteration = 35 Scaled Residual = 0.00760078 Iteration = 36 Scaled Residual = 0.00738145 Iteration = 37 Scaled Residual = 0.00717895 Iteration = 38 Scaled Residual = 0.0069928 Iteration = 39 Scaled Residual = 0.00682121 Iteration = 40 Scaled Residual = 0.00666289 Iteration = 41 Scaled Residual = 0.00651583 Iteration = 42 Scaled Residual = 0.00637718 Iteration = 43 Scaled Residual = 0.00624457 Iteration = 44 Scaled Residual = 0.0061155 Iteration = 45 Scaled Residual = 0.00598762 Iteration = 46 Scaled Residual = 0.00586007 Iteration = 47 Scaled Residual = 0.00573244 Iteration = 48 Scaled Residual = 0.00560537 Iteration = 49 Scaled Residual = 0.00548086 Iteration = 50 Scaled Residual = 0.00536085 Iteration = 51 Scaled Residual = 0.00524702 Iteration = 52 Scaled Residual = 0.0051409 Iteration = 53 Scaled Residual = 0.00504296 Iteration = 54 Scaled Residual = 0.00495328 Iteration = 55 Scaled Residual = 0.00487142 Call [0] Scaled Residual [0.00487142] Call [1] Scaled Residual [0.00487142] Call [2] Scaled Residual [0.00487142] Call [3] Scaled Residual [0.00487142] Call [4] Scaled Residual [0.00487142] Call [5] Scaled Residual [0.00487142] Call [6] Scaled Residual [0.00487142] Call [7] Scaled Residual [0.00487142] Call [8] Scaled Residual [0.00487142] Call [9] Scaled Residual [0.00487142] Call [10] Scaled Residual [0.00487142] Call [11] Scaled Residual [0.00487142] Call [12] Scaled Residual [0.00487142] Call [13] Scaled Residual [0.00487142] Call [14] Scaled Residual [0.00487142] Call [15] Scaled Residual [0.00487142] Call [16] Scaled Residual [0.00487142] Call [17] Scaled Residual [0.00487142] Call [18] Scaled Residual [0.00487142] Call [19] Scaled Residual [0.00487142] Call [20] Scaled Residual [0.00487142] Call [21] Scaled Residual [0.00487142] Call [22] Scaled Residual [0.00487142] Call [23] Scaled Residual [0.00487142] Call [24] Scaled Residual [0.00487142] Call [25] Scaled Residual [0.00487142] Call [26] Scaled Residual [0.00487142] Call [27] Scaled Residual [0.00487142] Call [28] Scaled Residual [0.00487142] Call [29] Scaled Residual [0.00487142] Call [30] Scaled Residual [0.00487142] Call [31] Scaled Residual [0.00487142] Call [32] Scaled Residual [0.00487142] Call [33] Scaled Residual [0.00487142] Call [34] Scaled Residual [0.00487142] Call [35] Scaled Residual [0.00487142] Call [36] Scaled Residual [0.00487142] Call [37] Scaled Residual [0.00487142] Call [38] Scaled Residual [0.00487142] Call [39] Scaled Residual [0.00487142] Call [40] Scaled Residual [0.00487142] Call [41] Scaled Residual [0.00487142] Call [42] Scaled Residual [0.00487142] Call [43] Scaled Residual [0.00487142] Call [44] Scaled Residual [0.00487142] Call [45] Scaled Residual [0.00487142] Call [46] Scaled Residual [0.00487142] Call [47] Scaled Residual [0.00487142] Call [48] Scaled Residual [0.00487142] Call [49] Scaled Residual [0.00487142] Call [50] Scaled Residual [0.00487142] Call [51] Scaled Residual [0.00487142] Call [52] Scaled Residual [0.00487142] Call [53] Scaled Residual [0.00487142] Call [54] Scaled Residual [0.00487142] Call [55] Scaled Residual [0.00487142] Call [56] Scaled Residual [0.00487142] Call [57] Scaled Residual [0.00487142] Call [58] Scaled Residual [0.00487142] Call [59] Scaled Residual [0.00487142] Call [60] Scaled Residual [0.00487142] Call [61] Scaled Residual [0.00487142] Call [62] Scaled Residual [0.00487142] Call [63] Scaled Residual [0.00487142] Call [64] Scaled Residual [0.00487142] Call [65] Scaled Residual [0.00487142] Call [66] Scaled Residual [0.00487142] Call [67] Scaled Residual [0.00487142] Call [68] Scaled Residual [0.00487142] Call [69] Scaled Residual [0.00487142] Call [70] Scaled Residual [0.00487142] Call [71] Scaled Residual [0.00487142] Call [72] Scaled Residual [0.00487142] Call [73] Scaled Residual [0.00487142] Call [74] Scaled Residual [0.00487142] Call [75] Scaled Residual [0.00487142] Call [76] Scaled Residual [0.00487142] Call [77] Scaled Residual [0.00487142] Call [78] Scaled Residual [0.00487142] Call [79] Scaled Residual [0.00487142] Call [80] Scaled Residual [0.00487142] Call [81] Scaled Residual [0.00487142] Call [82] Scaled Residual [0.00487142] Call [83] Scaled Residual [0.00487142] Call [84] Scaled Residual [0.00487142] Call [85] Scaled Residual [0.00487142] Call [86] Scaled Residual [0.00487142] Call [87] Scaled Residual [0.00487142] Call [88] Scaled Residual [0.00487142] Call [89] Scaled Residual [0.00487142] Call [90] Scaled Residual [0.00487142] Call [91] Scaled Residual [0.00487142] Call [92] Scaled Residual [0.00487142] Call [93] Scaled Residual [0.00487142] Call [94] Scaled Residual [0.00487142] Call [95] Scaled Residual [0.00487142] Call [96] Scaled Residual [0.00487142] Call [97] Scaled Residual [0.00487142] Call [98] Scaled Residual [0.00487142] Call [99] Scaled Residual [0.00487142] Call [100] Scaled Residual [0.00487142] Call [101] Scaled Residual [0.00487142] Call [102] Scaled Residual [0.00487142] Call [103] Scaled Residual [0.00487142] Call [104] Scaled Residual [0.00487142] Call [105] Scaled Residual [0.00487142] Call [106] Scaled Residual [0.00487142] Call [107] Scaled Residual [0.00487142] Call [108] Scaled Residual [0.00487142] Call [109] Scaled Residual [0.00487142] Call [110] Scaled Residual [0.00487142] Call [111] Scaled Residual [0.00487142] Call [112] Scaled Residual [0.00487142] Call [113] Scaled Residual [0.00487142] Call [114] Scaled Residual [0.00487142] Call [115] Scaled Residual [0.00487142] Call [116] Scaled Residual [0.00487142] Call [117] Scaled Residual [0.00487142] Call [118] Scaled Residual [0.00487142] Call [119] Scaled Residual [0.00487142] Call [120] Scaled Residual [0.00487142] Call [121] Scaled Residual [0.00487142] Call [122] Scaled Residual [0.00487142] Call [123] Scaled Residual [0.00487142] Call [124] Scaled Residual [0.00487142] Call [125] Scaled Residual [0.00487142] Call [126] Scaled Residual [0.00487142] Call [127] Scaled Residual [0.00487142] Call [128] Scaled Residual [0.00487142] Call [129] Scaled Residual [0.00487142] Call [130] Scaled Residual [0.00487142] Call [131] Scaled Residual [0.00487142] Call [132] Scaled Residual [0.00487142] Call [133] Scaled Residual [0.00487142] Call [134] Scaled Residual [0.00487142] Call [135] Scaled Residual [0.00487142] Call [136] Scaled Residual [0.00487142] Call [137] Scaled Residual [0.00487142] Call [138] Scaled Residual [0.00487142] Call [139] Scaled Residual [0.00487142] Call [140] Scaled Residual [0.00487142] Call [141] Scaled Residual [0.00487142] Call [142] Scaled Residual [0.00487142] Call [143] Scaled Residual [0.00487142] Call [144] Scaled Residual [0.00487142] Call [145] Scaled Residual [0.00487142] Call [146] Scaled Residual [0.00487142] Call [147] Scaled Residual [0.00487142] Call [148] Scaled Residual [0.00487142] Call [149] Scaled Residual [0.00487142] Call [150] Scaled Residual [0.00487142] Call [151] Scaled Residual [0.00487142] Call [152] Scaled Residual [0.00487142] Call [153] Scaled Residual [0.00487142] Call [154] Scaled Residual [0.00487142] Call [155] Scaled Residual [0.00487142] Call [156] Scaled Residual [0.00487142] Call [157] Scaled Residual [0.00487142] Call [158] Scaled Residual [0.00487142] Call [159] Scaled Residual [0.00487142] Call [160] Scaled Residual [0.00487142] Call [161] Scaled Residual [0.00487142] Call [162] Scaled Residual [0.00487142] Call [163] Scaled Residual [0.00487142] Call [164] Scaled Residual [0.00487142] Call [165] Scaled Residual [0.00487142] Call [166] Scaled Residual [0.00487142] Call [167] Scaled Residual [0.00487142] Call [168] Scaled Residual [0.00487142] Call [169] Scaled Residual [0.00487142] Call [170] Scaled Residual [0.00487142] Call [171] Scaled Residual [0.00487142] Call [172] Scaled Residual [0.00487142] Call [173] Scaled Residual [0.00487142] Call [174] Scaled Residual [0.00487142] Call [175] Scaled Residual [0.00487142] Call [176] Scaled Residual [0.00487142] Call [177] Scaled Residual [0.00487142] Call [178] Scaled Residual [0.00487142] Call [179] Scaled Residual [0.00487142] Call [180] Scaled Residual [0.00487142] Call [181] Scaled Residual [0.00487142] Call [182] Scaled Residual [0.00487142] Call [183] Scaled Residual [0.00487142] Call [184] Scaled Residual [0.00487142] Call [185] Scaled Residual [0.00487142] Call [186] Scaled Residual [0.00487142] Call [187] Scaled Residual [0.00487142] Call [188] Scaled Residual [0.00487142] Call [189] Scaled Residual [0.00487142] Call [190] Scaled Residual [0.00487142] Call [191] Scaled Residual [0.00487142] Call [192] Scaled Residual [0.00487142] Call [193] Scaled Residual [0.00487142] Call [194] Scaled Residual [0.00487142] Call [195] Scaled Residual [0.00487142] Call [196] Scaled Residual [0.00487142] Call [197] Scaled Residual [0.00487142] Call [198] Scaled Residual [0.00487142] Call [199] Scaled Residual [0.00487142] Call [200] Scaled Residual [0.00487142] Call [201] Scaled Residual [0.00487142] Call [202] Scaled Residual [0.00487142] Call [203] Scaled Residual [0.00487142] Call [204] Scaled Residual [0.00487142] Call [205] Scaled Residual [0.00487142] Call [206] Scaled Residual [0.00487142] Call [207] Scaled Residual [0.00487142] Call [208] Scaled Residual [0.00487142] Call [209] Scaled Residual [0.00487142] Call [210] Scaled Residual [0.00487142] Call [211] Scaled Residual [0.00487142] Call [212] Scaled Residual [0.00487142] Call [213] Scaled Residual [0.00487142] Call [214] Scaled Residual [0.00487142] Call [215] Scaled Residual [0.00487142] Call [216] Scaled Residual [0.00487142] Call [217] Scaled Residual [0.00487142] Call [218] Scaled Residual [0.00487142] Call [219] Scaled Residual [0.00487142] Call [220] Scaled Residual [0.00487142] Call [221] Scaled Residual [0.00487142] Call [222] Scaled Residual [0.00487142] Call [223] Scaled Residual [0.00487142] Call [224] Scaled Residual [0.00487142] Call [225] Scaled Residual [0.00487142] Call [226] Scaled Residual [0.00487142] Call [227] Scaled Residual [0.00487142] Call [228] Scaled Residual [0.00487142] Call [229] Scaled Residual [0.00487142] Call [230] Scaled Residual [0.00487142] Call [231] Scaled Residual [0.00487142] Call [232] Scaled Residual [0.00487142] Call [233] Scaled Residual [0.00487142] Call [234] Scaled Residual [0.00487142] Call [235] Scaled Residual [0.00487142] Call [236] Scaled Residual [0.00487142] Call [237] Scaled Residual [0.00487142] Call [238] Scaled Residual [0.00487142] Call [239] Scaled Residual [0.00487142] Call [240] Scaled Residual [0.00487142] Call [241] Scaled Residual [0.00487142] Call [242] Scaled Residual [0.00487142] Call [243] Scaled Residual [0.00487142] Call [244] Scaled Residual [0.00487142] Call [245] Scaled Residual [0.00487142] Call [246] Scaled Residual [0.00487142] Call [247] Scaled Residual [0.00487142] Call [248] Scaled Residual [0.00487142] Call [249] Scaled Residual [0.00487142] Call [250] Scaled Residual [0.00487142] Call [251] Scaled Residual [0.00487142] Call [252] Scaled Residual [0.00487142] Call [253] Scaled Residual [0.00487142] Call [254] Scaled Residual [0.00487142] Call [255] Scaled Residual [0.00487142] Call [256] Scaled Residual [0.00487142] Call [257] Scaled Residual [0.00487142] Call [258] Scaled Residual [0.00487142] Call [259] Scaled Residual [0.00487142] Call [260] Scaled Residual [0.00487142] Call [261] Scaled Residual [0.00487142] Call [262] Scaled Residual [0.00487142] Call [263] Scaled Residual [0.00487142] Call [264] Scaled Residual [0.00487142] Call [265] Scaled Residual [0.00487142] Call [266] Scaled Residual [0.00487142] Call [267] Scaled Residual [0.00487142] Call [268] Scaled Residual [0.00487142] Call [269] Scaled Residual [0.00487142] Call [270] Scaled Residual [0.00487142] Call [271] Scaled Residual [0.00487142] Call [272] Scaled Residual [0.00487142] Call [273] Scaled Residual [0.00487142] Call [274] Scaled Residual [0.00487142] Call [275] Scaled Residual [0.00487142] Call [276] Scaled Residual [0.00487142] Call [277] Scaled Residual [0.00487142] Call [278] Scaled Residual [0.00487142] Call [279] Scaled Residual [0.00487142] Call [280] Scaled Residual [0.00487142] Call [281] Scaled Residual [0.00487142] Call [282] Scaled Residual [0.00487142] Call [283] Scaled Residual [0.00487142] Call [284] Scaled Residual [0.00487142] Call [285] Scaled Residual [0.00487142] Call [286] Scaled Residual [0.00487142] Call [287] Scaled Residual [0.00487142] Call [288] Scaled Residual [0.00487142] Call [289] Scaled Residual [0.00487142] Call [290] Scaled Residual [0.00487142] Call [291] Scaled Residual [0.00487142] Call [292] Scaled Residual [0.00487142] Call [293] Scaled Residual [0.00487142] Call [294] Scaled Residual [0.00487142] Call [295] Scaled Residual [0.00487142] Call [296] Scaled Residual [0.00487142] Call [297] Scaled Residual [0.00487142] Call [298] Scaled Residual [0.00487142] Call [299] Scaled Residual [0.00487142] Call [300] Scaled Residual [0.00487142] Call [301] Scaled Residual [0.00487142] Call [302] Scaled Residual [0.00487142] Call [303] Scaled Residual [0.00487142] Call [304] Scaled Residual [0.00487142] Call [305] Scaled Residual [0.00487142] Call [306] Scaled Residual [0.00487142] Call [307] Scaled Residual [0.00487142] Call [308] Scaled Residual [0.00487142] Call [309] Scaled Residual [0.00487142] Call [310] Scaled Residual [0.00487142] Call [311] Scaled Residual [0.00487142] Call [312] Scaled Residual [0.00487142] Call [313] Scaled Residual [0.00487142] Call [314] Scaled Residual [0.00487142] Call [315] Scaled Residual [0.00487142] Call [316] Scaled Residual [0.00487142] Call [317] Scaled Residual [0.00487142] Call [318] Scaled Residual [0.00487142] Call [319] Scaled Residual [0.00487142] Call [320] Scaled Residual [0.00487142] Call [321] Scaled Residual [0.00487142] Call [322] Scaled Residual [0.00487142] Call [323] Scaled Residual [0.00487142] Call [324] Scaled Residual [0.00487142] Call [325] Scaled Residual [0.00487142] Call [326] Scaled Residual [0.00487142] Call [327] Scaled Residual [0.00487142] Call [328] Scaled Residual [0.00487142] Call [329] Scaled Residual [0.00487142] Call [330] Scaled Residual [0.00487142] Call [331] Scaled Residual [0.00487142] Call [332] Scaled Residual [0.00487142] Call [333] Scaled Residual [0.00487142] Call [334] Scaled Residual [0.00487142] Call [335] Scaled Residual [0.00487142] Call [336] Scaled Residual [0.00487142] Call [337] Scaled Residual [0.00487142] Call [338] Scaled Residual [0.00487142] Call [339] Scaled Residual [0.00487142] Call [340] Scaled Residual [0.00487142] Call [341] Scaled Residual [0.00487142] Call [342] Scaled Residual [0.00487142] Call [343] Scaled Residual [0.00487142] Call [344] Scaled Residual [0.00487142] Call [345] Scaled Residual [0.00487142] Call [346] Scaled Residual [0.00487142] Call [347] Scaled Residual [0.00487142] Call [348] Scaled Residual [0.00487142] Call [349] Scaled Residual [0.00487142] Call [350] Scaled Residual [0.00487142] Call [351] Scaled Residual [0.00487142] Call [352] Scaled Residual [0.00487142] Call [353] Scaled Residual [0.00487142] Call [354] Scaled Residual [0.00487142] Call [355] Scaled Residual [0.00487142] Call [356] Scaled Residual [0.00487142] Call [357] Scaled Residual [0.00487142] Call [358] Scaled Residual [0.00487142] Call [359] Scaled Residual [0.00487142] Call [360] Scaled Residual [0.00487142] Call [361] Scaled Residual [0.00487142] Call [362] Scaled Residual [0.00487142] Call [363] Scaled Residual [0.00487142] Call [364] Scaled Residual [0.00487142] Call [365] Scaled Residual [0.00487142] Call [366] Scaled Residual [0.00487142] Call [367] Scaled Residual [0.00487142] Call [368] Scaled Residual [0.00487142] Call [369] Scaled Residual [0.00487142] Call [370] Scaled Residual [0.00487142] Call [371] Scaled Residual [0.00487142] Call [372] Scaled Residual [0.00487142] Call [373] Scaled Residual [0.00487142] Call [374] Scaled Residual [0.00487142] Call [375] Scaled Residual [0.00487142] Call [376] Scaled Residual [0.00487142] Call [377] Scaled Residual [0.00487142] Call [378] Scaled Residual [0.00487142] Call [379] Scaled Residual [0.00487142] Call [380] Scaled Residual [0.00487142] Call [381] Scaled Residual [0.00487142] Call [382] Scaled Residual [0.00487142] Call [383] Scaled Residual [0.00487142] Call [384] Scaled Residual [0.00487142] Call [385] Scaled Residual [0.00487142] Call [386] Scaled Residual [0.00487142] Call [387] Scaled Residual [0.00487142] Call [388] Scaled Residual [0.00487142] Call [389] Scaled Residual [0.00487142] Call [390] Scaled Residual [0.00487142] Call [391] Scaled Residual [0.00487142] Call [392] Scaled Residual [0.00487142] Call [393] Scaled Residual [0.00487142] Call [394] Scaled Residual [0.00487142] Call [395] Scaled Residual [0.00487142] Call [396] Scaled Residual [0.00487142] Call [397] Scaled Residual [0.00487142] Call [398] Scaled Residual [0.00487142] Call [399] Scaled Residual [0.00487142] Call [400] Scaled Residual [0.00487142] Call [401] Scaled Residual [0.00487142] Call [402] Scaled Residual [0.00487142] Call [403] Scaled Residual [0.00487142] Call [404] Scaled Residual [0.00487142] Call [405] Scaled Residual [0.00487142] Call [406] Scaled Residual [0.00487142] Call [407] Scaled Residual [0.00487142] Call [408] Scaled Residual [0.00487142] Call [409] Scaled Residual [0.00487142] Call [410] Scaled Residual [0.00487142] Call [411] Scaled Residual [0.00487142] Call [412] Scaled Residual [0.00487142] Call [413] Scaled Residual [0.00487142] Call [414] Scaled Residual [0.00487142] Call [415] Scaled Residual [0.00487142] Call [416] Scaled Residual [0.00487142] Call [417] Scaled Residual [0.00487142] Call [418] Scaled Residual [0.00487142] Call [419] Scaled Residual [0.00487142] Call [420] Scaled Residual [0.00487142] Call [421] Scaled Residual [0.00487142] Call [422] Scaled Residual [0.00487142] Call [423] Scaled Residual [0.00487142] Call [424] Scaled Residual [0.00487142] Call [425] Scaled Residual [0.00487142] Call [426] Scaled Residual [0.00487142] Call [427] Scaled Residual [0.00487142] Call [428] Scaled Residual [0.00487142] Call [429] Scaled Residual [0.00487142] Call [430] Scaled Residual [0.00487142] Call [431] Scaled Residual [0.00487142] Call [432] Scaled Residual [0.00487142] Call [433] Scaled Residual [0.00487142] Call [434] Scaled Residual [0.00487142] Call [435] Scaled Residual [0.00487142] Call [436] Scaled Residual [0.00487142] Call [437] Scaled Residual [0.00487142] Call [438] Scaled Residual [0.00487142] Call [439] Scaled Residual [0.00487142] Call [440] Scaled Residual [0.00487142] Call [441] Scaled Residual [0.00487142] Call [442] Scaled Residual [0.00487142] Call [443] Scaled Residual [0.00487142] Call [444] Scaled Residual [0.00487142] Call [445] Scaled Residual [0.00487142] Call [446] Scaled Residual [0.00487142] Call [447] Scaled Residual [0.00487142] Call [448] Scaled Residual [0.00487142] Call [449] Scaled Residual [0.00487142] Call [450] Scaled Residual [0.00487142] Call [451] Scaled Residual [0.00487142] Call [452] Scaled Residual [0.00487142] Call [453] Scaled Residual [0.00487142] Call [454] Scaled Residual [0.00487142] Call [455] Scaled Residual [0.00487142] Call [456] Scaled Residual [0.00487142] Call [457] Scaled Residual [0.00487142] Call [458] Scaled Residual [0.00487142] Call [459] Scaled Residual [0.00487142] Call [460] Scaled Residual [0.00487142] Call [461] Scaled Residual [0.00487142] Call [462] Scaled Residual [0.00487142] Call [463] Scaled Residual [0.00487142] Call [464] Scaled Residual [0.00487142] Call [465] Scaled Residual [0.00487142] Call [466] Scaled Residual [0.00487142] Call [467] Scaled Residual [0.00487142] Call [468] Scaled Residual [0.00487142] Call [469] Scaled Residual [0.00487142] Call [470] Scaled Residual [0.00487142] Call [471] Scaled Residual [0.00487142] Call [472] Scaled Residual [0.00487142] Call [473] Scaled Residual [0.00487142] Call [474] Scaled Residual [0.00487142] Call [475] Scaled Residual [0.00487142] Call [476] Scaled Residual [0.00487142] Call [477] Scaled Residual [0.00487142] Call [478] Scaled Residual [0.00487142] Call [479] Scaled Residual [0.00487142] Call [480] Scaled Residual [0.00487142] Call [481] Scaled Residual [0.00487142] Call [482] Scaled Residual [0.00487142] Call [483] Scaled Residual [0.00487142] Call [484] Scaled Residual [0.00487142] Call [485] Scaled Residual [0.00487142] Call [486] Scaled Residual [0.00487142] Call [487] Scaled Residual [0.00487142] Call [488] Scaled Residual [0.00487142] Call [489] Scaled Residual [0.00487142] Call [490] Scaled Residual [0.00487142] Call [491] Scaled Residual [0.00487142] Call [492] Scaled Residual [0.00487142] Call [493] Scaled Residual [0.00487142] Call [494] Scaled Residual [0.00487142] Call [495] Scaled Residual [0.00487142] Call [496] Scaled Residual [0.00487142] Call [497] Scaled Residual [0.00487142] Call [498] Scaled Residual [0.00487142] Call [499] Scaled Residual [0.00487142] Call [500] Scaled Residual [0.00487142] Call [501] Scaled Residual [0.00487142] Call [502] Scaled Residual [0.00487142] Call [503] Scaled Residual [0.00487142] Call [504] Scaled Residual [0.00487142] Call [505] Scaled Residual [0.00487142] Call [506] Scaled Residual [0.00487142] Call [507] Scaled Residual [0.00487142] Call [508] Scaled Residual [0.00487142] Call [509] Scaled Residual [0.00487142] Call [510] Scaled Residual [0.00487142] Call [511] Scaled Residual [0.00487142] Call [512] Scaled Residual [0.00487142] Call [513] Scaled Residual [0.00487142] Call [514] Scaled Residual [0.00487142] Call [515] Scaled Residual [0.00487142] Call [516] Scaled Residual [0.00487142] Call [517] Scaled Residual [0.00487142] Call [518] Scaled Residual [0.00487142] Call [519] Scaled Residual [0.00487142] Call [520] Scaled Residual [0.00487142] Call [521] Scaled Residual [0.00487142] Call [522] Scaled Residual [0.00487142] Call [523] Scaled Residual [0.00487142] Call [524] Scaled Residual [0.00487142] Call [525] Scaled Residual [0.00487142] Call [526] Scaled Residual [0.00487142] Call [527] Scaled Residual [0.00487142] Call [528] Scaled Residual [0.00487142] Call [529] Scaled Residual [0.00487142] Call [530] Scaled Residual [0.00487142] Call [531] Scaled Residual [0.00487142] Call [532] Scaled Residual [0.00487142] Call [533] Scaled Residual [0.00487142] Call [534] Scaled Residual [0.00487142] Call [535] Scaled Residual [0.00487142] Call [536] Scaled Residual [0.00487142] Call [537] Scaled Residual [0.00487142] Call [538] Scaled Residual [0.00487142] Call [539] Scaled Residual [0.00487142] Call [540] Scaled Residual [0.00487142] Call [541] Scaled Residual [0.00487142] Call [542] Scaled Residual [0.00487142] Call [543] Scaled Residual [0.00487142] Call [544] Scaled Residual [0.00487142] Call [545] Scaled Residual [0.00487142] Call [546] Scaled Residual [0.00487142] Call [547] Scaled Residual [0.00487142] Call [548] Scaled Residual [0.00487142] Call [549] Scaled Residual [0.00487142] Call [550] Scaled Residual [0.00487142] Call [551] Scaled Residual [0.00487142] Call [552] Scaled Residual [0.00487142] Call [553] Scaled Residual [0.00487142] Call [554] Scaled Residual [0.00487142] Call [555] Scaled Residual [0.00487142] Call [556] Scaled Residual [0.00487142] Call [557] Scaled Residual [0.00487142] Call [558] Scaled Residual [0.00487142] Call [559] Scaled Residual [0.00487142] Call [560] Scaled Residual [0.00487142] Call [561] Scaled Residual [0.00487142] Call [562] Scaled Residual [0.00487142] Call [563] Scaled Residual [0.00487142] Call [564] Scaled Residual [0.00487142] Call [565] Scaled Residual [0.00487142] Call [566] Scaled Residual [0.00487142] Call [567] Scaled Residual [0.00487142] Call [568] Scaled Residual [0.00487142] Call [569] Scaled Residual [0.00487142] Call [570] Scaled Residual [0.00487142] Call [571] Scaled Residual [0.00487142] Call [572] Scaled Residual [0.00487142] Call [573] Scaled Residual [0.00487142] Call [574] Scaled Residual [0.00487142] Call [575] Scaled Residual [0.00487142] Call [576] Scaled Residual [0.00487142] Call [577] Scaled Residual [0.00487142] Call [578] Scaled Residual [0.00487142] Call [579] Scaled Residual [0.00487142] Call [580] Scaled Residual [0.00487142] Call [581] Scaled Residual [0.00487142] Call [582] Scaled Residual [0.00487142] Call [583] Scaled Residual [0.00487142] Call [584] Scaled Residual [0.00487142] Call [585] Scaled Residual [0.00487142] Call [586] Scaled Residual [0.00487142] Call [587] Scaled Residual [0.00487142] Call [588] Scaled Residual [0.00487142] Call [589] Scaled Residual [0.00487142] Call [590] Scaled Residual [0.00487142] Call [591] Scaled Residual [0.00487142] Call [592] Scaled Residual [0.00487142] Call [593] Scaled Residual [0.00487142] Call [594] Scaled Residual [0.00487142] Call [595] Scaled Residual [0.00487142] Call [596] Scaled Residual [0.00487142] Call [597] Scaled Residual [0.00487142] Call [598] Scaled Residual [0.00487142] Call [599] Scaled Residual [0.00487142] Call [600] Scaled Residual [0.00487142] Call [601] Scaled Residual [0.00487142] Call [602] Scaled Residual [0.00487142] Call [603] Scaled Residual [0.00487142] Call [604] Scaled Residual [0.00487142] Call [605] Scaled Residual [0.00487142] Call [606] Scaled Residual [0.00487142] Call [607] Scaled Residual [0.00487142] Call [608] Scaled Residual [0.00487142] Call [609] Scaled Residual [0.00487142] Call [610] Scaled Residual [0.00487142] Call [611] Scaled Residual [0.00487142] Call [612] Scaled Residual [0.00487142] Call [613] Scaled Residual [0.00487142] Call [614] Scaled Residual [0.00487142] Call [615] Scaled Residual [0.00487142] Call [616] Scaled Residual [0.00487142] Call [617] Scaled Residual [0.00487142] Call [618] Scaled Residual [0.00487142] Call [619] Scaled Residual [0.00487142] Call [620] Scaled Residual [0.00487142] Call [621] Scaled Residual [0.00487142] Call [622] Scaled Residual [0.00487142] Call [623] Scaled Residual [0.00487142] Call [624] Scaled Residual [0.00487142] Call [625] Scaled Residual [0.00487142] Call [626] Scaled Residual [0.00487142] Call [627] Scaled Residual [0.00487142] Call [628] Scaled Residual [0.00487142] Call [629] Scaled Residual [0.00487142] Call [630] Scaled Residual [0.00487142] Call [631] Scaled Residual [0.00487142] Call [632] Scaled Residual [0.00487142] Call [633] Scaled Residual [0.00487142] Call [634] Scaled Residual [0.00487142] Call [635] Scaled Residual [0.00487142] Call [636] Scaled Residual [0.00487142] Call [637] Scaled Residual [0.00487142] Call [638] Scaled Residual [0.00487142] Call [639] Scaled Residual [0.00487142] Call [640] Scaled Residual [0.00487142] Call [641] Scaled Residual [0.00487142] Call [642] Scaled Residual [0.00487142] Call [643] Scaled Residual [0.00487142] Call [644] Scaled Residual [0.00487142] Call [645] Scaled Residual [0.00487142] Call [646] Scaled Residual [0.00487142] Call [647] Scaled Residual [0.00487142] Call [648] Scaled Residual [0.00487142] Call [649] Scaled Residual [0.00487142] Call [650] Scaled Residual [0.00487142] Call [651] Scaled Residual [0.00487142] Call [652] Scaled Residual [0.00487142] Call [653] Scaled Residual [0.00487142] Call [654] Scaled Residual [0.00487142] Call [655] Scaled Residual [0.00487142] Call [656] Scaled Residual [0.00487142] Call [657] Scaled Residual [0.00487142] Call [658] Scaled Residual [0.00487142] Call [659] Scaled Residual [0.00487142] Call [660] Scaled Residual [0.00487142] Call [661] Scaled Residual [0.00487142] Call [662] Scaled Residual [0.00487142] Call [663] Scaled Residual [0.00487142] Call [664] Scaled Residual [0.00487142] Call [665] Scaled Residual [0.00487142] Call [666] Scaled Residual [0.00487142] Call [667] Scaled Residual [0.00487142] Call [668] Scaled Residual [0.00487142] Call [669] Scaled Residual [0.00487142] Call [670] Scaled Residual [0.00487142] Call [671] Scaled Residual [0.00487142] Call [672] Scaled Residual [0.00487142] Call [673] Scaled Residual [0.00487142] Call [674] Scaled Residual [0.00487142] Call [675] Scaled Residual [0.00487142] Call [676] Scaled Residual [0.00487142] Call [677] Scaled Residual [0.00487142] Call [678] Scaled Residual [0.00487142] HPCG-Benchmark version=3.1 Release date=March 28, 2019 Machine Summary= Machine Summary::Distributed Processes=1856 Machine Summary::Threads per processes=72 Global Problem Dimensions= Global Problem Dimensions::Global nx=4096 Global Problem Dimensions::Global ny=4096 Global Problem Dimensions::Global nz=8352 Processor Dimensions= Processor Dimensions::npx=8 Processor Dimensions::npy=8 Processor Dimensions::npz=29 Local Domain Dimensions= Local Domain Dimensions::nx=512 Local Domain Dimensions::ny=512 ########## Problem Summary ##########= Setup Information= Setup Information::Setup Time=0.275541 Linear System Information= Linear System Information::Number of Equations=140123308032 Linear System Information::Number of Nonzero Terms=3781795972984 Multigrid Information= Multigrid Information::Number of coarse grid levels=3 Multigrid Information::Coarse Grids= Multigrid Information::Coarse Grids::Grid Level=1 Multigrid Information::Coarse Grids::Number of Equations=17515413504 Multigrid Information::Coarse Grids::Number of Nonzero Terms=472532878264 Multigrid Information::Coarse Grids::Number of Presmoother Steps=1 Multigrid Information::Coarse Grids::Number of Postsmoother Steps=1 Multigrid Information::Coarse Grids::Grid Level=2 Multigrid Information::Coarse Grids::Number of Equations=2189426688 Multigrid Information::Coarse Grids::Number of Nonzero Terms=59018723800 Multigrid Information::Coarse Grids::Number of Presmoother Steps=1 Multigrid Information::Coarse Grids::Number of Postsmoother Steps=1 Multigrid Information::Coarse Grids::Grid Level=3 Multigrid Information::Coarse Grids::Number of Equations=273678336 Multigrid Information::Coarse Grids::Number of Nonzero Terms=7365378280 Multigrid Information::Coarse Grids::Number of Presmoother Steps=1 Multigrid Information::Coarse Grids::Number of Postsmoother Steps=1 ########## Memory Use Summary ##########= Memory Use Information= Memory Use Information::Total memory used for data (Gbytes)=100167 Memory Use Information::Memory used for OptimizeProblem data (Gbytes)=0 Memory Use Information::Bytes per equation (Total memory / Number of Equations)=714.847 Memory Use Information::Memory used for linear system and CG (Gbytes)=88154.1 Memory Use Information::Coarse Grids= Memory Use Information::Coarse Grids::Grid Level=1 Memory Use Information::Coarse Grids::Memory used=10530.9 Memory Use Information::Coarse Grids::Grid Level=2 Memory Use Information::Coarse Grids::Memory used=1316.89 Memory Use Information::Coarse Grids::Grid Level=3 Memory Use Information::Coarse Grids::Memory used=164.744 ########## V&V Testing Summary ##########= Spectral Convergence Tests= Spectral Convergence Tests::Result=PASSED Spectral Convergence Tests::Unpreconditioned= Spectral Convergence Tests::Unpreconditioned::Maximum iteration count=11 Spectral Convergence Tests::Unpreconditioned::Expected iteration count=12 Spectral Convergence Tests::Preconditioned= Spectral Convergence Tests::Preconditioned::Maximum iteration count=2 Spectral Convergence Tests::Preconditioned::Expected iteration count=2 Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon= Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon::Result=PASSED Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon::Departure for SpMV=1.93174e-16 Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon::Departure for MG=0 ########## Iterations Summary ##########= Iteration Count Information= Iteration Count Information::Result=PASSED Iteration Count Information::Reference CG iterations per set=50 Iteration Count Information::Optimized CG iterations per set=55 Iteration Count Information::Total number of reference iterations=33950 Iteration Count Information::Total number of optimized iterations=37345 ########## Reproducibility Summary ##########= Reproducibility Information= Reproducibility Information::Result=PASSED Reproducibility Information::Scaled residual mean=0.00487142 Reproducibility Information::Scaled residual variance=0 ########## Performance Summary (times in sec) ##########= Benchmark Time Summary= Benchmark Time Summary::Optimization phase=0.249756 Benchmark Time Summary::DDOT=339.471 Benchmark Time Summary::WAXPBY=55.1415 Benchmark Time Summary::SpMV=272.672 Benchmark Time Summary::MG=1185.97 Benchmark Time Summary::Total=1853.27 Floating Point Operations Summary= Floating Point Operations Summary::Raw DDOT=3.15877e+16 Floating Point Operations Summary::Raw WAXPBY=3.15877e+16 Floating Point Operations Summary::Raw SpMV=2.87598e+17 Floating Point Operations Summary::Raw MG=1.61192e+18 Floating Point Operations Summary::Total=1.96269e+18 Floating Point Operations Summary::Total with convergence overhead=1.78427e+18 GB/s Summary= GB/s Summary::Raw Read B/W=6.52277e+06 GB/s Summary::Raw Write B/W=1.50767e+06 GB/s Summary::Raw Total B/W=8.03044e+06 GB/s Summary::Total with convergence and optimization phase overhead=7.16255e+06 GFLOP/s Summary= GFLOP/s Summary::Raw DDOT=93049.9 GFLOP/s Summary::Raw WAXPBY=572848 GFLOP/s Summary::Raw SpMV=1.05474e+06 GFLOP/s Summary::Raw MG=1.35916e+06 GFLOP/s Summary::Raw Total=1.05904e+06 GFLOP/s Summary::Total with convergence overhead=962766 GFLOP/s Summary::Total with convergence and optimization phase overhead=944586 User Optimization Overheads= User Optimization Overheads::Optimization phase time (sec)=0.249756 User Optimization Overheads::Optimization phase time vs reference SpMV+MG time=0.0351581 DDOT Timing Variations= DDOT Timing Variations::Min DDOT MPI_Allreduce time=233.598 DDOT Timing Variations::Max DDOT MPI_Allreduce time=305.803 DDOT Timing Variations::Avg DDOT MPI_Allreduce time=252.324 Final Summary= Final Summary::HPCG result is VALID with a GFLOP/s rating of=944586 Final Summary::HPCG 2.4 rating for historical reasons is=954036 Final Summary::Please upload results from the YAML file contents to=http://hpcg-benchmark.org