STRUMPACK vs EIGEN performance
The goal of this notebook is to allow the documentation of STRUMPACK vs EIGEN performance to be maintained in a single accessible location. The environment within which this notebook is run follows the standard cctbx conda build instructions available here. For this instance, we are using the STRUMPACK-enabled build of cctbx located at ExaFEL:cctbx_project(str_merge). STRUMPACK is currently built using the installation script STRUMPACK_installer_shared.sh, and if the installation takes place within the same directory as moddules and build, the cctbx build process can make use of it as a backend. After the STRUMPACK solver finishes, enter the build directory, run libtbx.refresh and make. The STRUMPACK-supported modules should now build and link with the new backend.
The solution of the below systems assume that we have some A and b data already, and we can provide these in CSV file format to the solver (A is in “row col value” format). We can begin by importing all of the required modules. We make use of Numpy’s ability to parse CSV files into numeric format, and SciPy’s sparse storage format to format the matrices into the required CSR sparse storage format for solving.
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import matplotlib
import matplotlib.pyplot as plt
font = {'family' : 'serif',
#'weight' : 'bold',
'size' : 12}
matplotlib.rc('font', **font)
matplotlib.rcParams['figure.figsize'] = (16,10)
matplotlib.rcParams['figure.dpi']= 150
import matplotlib.gridspec as gridspec
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
%env BOOST_ADAPTBX_FPE_DEFAULT=1
%env BOOST_ADAPTBX_SIGNALS_DEFAULT=1
env: BOOST_ADAPTBX_FPE_DEFAULT=1
env: BOOST_ADAPTBX_SIGNALS_DEFAULT=1
We load the A and b data set for solving from the locations specified below. Ideally, we can loop over and load all different data sets and process each individually. For a working example, we begin by processing a single data set, comparing Eigen and Strumpack for varying levels of shared parallelism.
A_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
b_vec = np.loadtxt(b_path)
We are solving system of normal equations, and so we can safely assume that we have a square matrix. This ensures the number of rows and colums will be identical, which can be read from the length of the numpy array b_vec.
n_rows = len(b_vec)
n_cols = n_rows
We next wish to create a SciPy CSR sparse matrix type using the data from A_mat.
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
With this, we can now create the appropriate flex data types for solving the linear system. While the operations performed upto and including this point involve many copy steps, it can be stated that the time to solve the system is much greater than the time to perform internal copies, and so we can ignore the times required for these steps for any realistic datasets.
For the solvers we require the row index pointers, the column indices, and sparse matrix values. However, it is import to first ensure that our A matrix is symmetric, and not the upper or lower triangle only. While Eigen’s Cholesky deocmposition can use the upper or lower triangle, other algorithms and solvers tend to expect a full matrix. We can create this as follows.
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_sp = A_sp + A_sp.transpose() - sd
plt.spy(A_sp, marker='.',markersize=1.0)
plt.show()
 |
The resulting data sets can be read directly and converted to flex types as follows.
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
With the data in the required format, we can now create a solver object and solve the system. We also include a built-in profiler object to time this solver step for both the STRUMPACK and EIGEN backends.
Next, we load the OpenMP-enabled STRUMPACK solver module and the Eigen based solver. These are contained within “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext.cpp” and where a Boost.Python wrapper for both an EIGEN and STRUMPACK Ax=b solver object is provided.
A non-notebook script exists to test this functionality at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver.py”, which can be called as:
OMP_NUM_THREADS=x libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
where A_mat.csv and b_vec.csv are the CSV files with the linear system to be solved, and x specifies the number of OpenMP threads to spawn. Additionally, the MPI-enabled STRUMPACK backend is provided by “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext_mpi_dist.cpp”, with the Python script to test the result located at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py”. This is called as
OMP_NUM_THREADS=x mpirun -n y libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
As an example, the previously loaded can be provided to the solvers as follows:
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
ssmd = ext_mpi.strumpack_mpi_dist_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b):
P = Profiler("STRUMPACK_SCOTCH")
res_strum_sc = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS")
res_strum_me = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
return res_strum_sc, res_strum_me, res_eig_ldlt, res_eig_bicgstab
r_s_sc, r_s_me, r_e_ldlt, r_e_bicgstab = run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b)
for i in xrange(len(r_s_sc.x)):
assert( approx_equal(r_s_sc.x[i], r_s_me.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_ldlt.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_bicgstab.x[i]) )
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
individual call time for STRUMPACK_SCOTCH: CPU, 19.210s; elapsed, 0.424s
individual call time for STRUMPACK_METIS: CPU, 176.190s; elapsed, 2.805s
individual call time for EIGEN_LDLT: CPU, 3.940s; elapsed, 0.063s
individual call time for EIGEN_BICGSTAB: CPU, 1.400s; elapsed, 0.023s
While we can use the above method to process the data, we must respecify the OMP_NUM_THREADS variable to observe different levels of scalability. %env OMP_NUM_THREADS is sufficient to test this for the first run, however for an undetermined reason the notebook must be started to allow the value to be changed. There, we can run the above tasks from a !<command> cell, specifying the thread count here instead.
OMP_SOLVER='''
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
import matplotlib
import matplotlib.pyplot as plt
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['figure.figsize'] = (16,10)
A_path=sys.argv[1]
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path=sys.argv[2]
b_vec = np.loadtxt(b_path)
n_rows = len(b_vec)
n_cols = n_rows
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check for triangular matrix. If so, A_sp := A+A^T - diag(A)
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
#import time
#timing_dict = {"strum":0, "eigen":0}
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS):
P = Profiler("EIGEN_LLT_CHOL")
res_eig_llt_chol = es(0, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_LDLT_CHOL")
res_eig_ldlt_chol = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
A_indptr = flex.int(A_spS.indptr)
A_indices = flex.int(A_spS.indices)
A_values = flex.double(A_spS.data)
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("STRUMPACK_SCOTCH_AUTO")
res_strum_sc_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS_AUTO")
res_strum_mt_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("STRUMPACK_SCOTCH_BICGSTAB")
res_strum_sc_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_METIS_BICGSTAB")
res_strum_mt_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_SCOTCH_PRECBICGSTAB")
res_strum_sc_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.prec_bicgstab)
del P
P = Profiler("STRUMPACK_METIS_PRECBICGSTAB")
res_strum_mt_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.prec_bicgstab)
del P
for i in xrange(len(res_strum_sc.x)):
assert( approx_equal(res_eig_llt_chol.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_bicgstab.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_a.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_pr.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_pr.x[i]) )
run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)
'''
OMP_SOLVER = OMP_SOLVER
OMP_SOLVER_FILE = open("OMP_SOLVER.py", "w")
OMP_SOLVER_FILE.write(OMP_SOLVER)
OMP_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We now record the indices of the data with a specific number of images. We can use these indices to later submit jobs with a given number of frames, and hence resulting matrix size.
list_idx={}
for imgs in ['1k','5k','10k','32k']:
list_idx.update({imgs:[i for i, j in enumerate(A_LIST) if imgs in j]})
list_idx
{'10k': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
'1k': [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
'32k': [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44],
'5k': [45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]}
With the file writtent to disk, we now create a loop in mixed Python/Bash to call the solver, then extract the standard output from the profiler to examine the OpenMP scalability of STRUMPACK. The profiler code wraps each solver, and does not take into account the load-times for the data.
str_out={}
import os
threads_list = [16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "1k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for threads in threads_list:
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} libtbx.python ./OMP_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting")
key = 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s, #calls: 1', 'TOTAL : CPU, 77.570s; elapsed, 6.455s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s, #calls: 1', 'TOTAL : CPU, 81.130s; elapsed, 5.566s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s, #calls: 1', 'TOTAL : CPU, 69.570s; elapsed, 4.823s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s, #calls: 1', 'TOTAL : CPU, 65.610s; elapsed, 4.247s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s, #calls: 1', 'TOTAL : CPU, 96.220s; elapsed, 9.909s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s, #calls: 1', 'TOTAL : CPU, 88.400s; elapsed, 9.434s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s, #calls: 1', 'TOTAL : CPU, 74.030s; elapsed, 6.238s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s, #calls: 1', 'TOTAL : CPU, 86.900s; elapsed, 7.087s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s, #calls: 1', 'TOTAL : CPU, 80.710s; elapsed, 5.525s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s, #calls: 1', 'TOTAL : CPU, 68.820s; elapsed, 4.782s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s, #calls: 1', 'TOTAL : CPU, 106.000s; elapsed, 14.140s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s, #calls: 1', 'TOTAL : CPU, 98.830s; elapsed, 10.102s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s, #calls: 1', 'TOTAL : CPU, 85.860s; elapsed, 9.266s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s, #calls: 1', 'TOTAL : CPU, 86.730s; elapsed, 7.075s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s, #calls: 1', 'TOTAL : CPU, 106.950s; elapsed, 14.208s']
Data Set Size:=5k
Skipping 5k
Data Set Size:=32k
Skipping 32k
In the event of a crash, it is wise to save the resulting data (JSON and Pickle) to disk…
import json, cPickle
with open("OMP_SOLVER_TIME_1k_all.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_1k_all.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
…which can then be easily reloaded later. Now, we parse the string, extract the timing data, and add it to the python object at the specified thread number index. This allows for easily sorting and plot the data later.
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_1k_all.pickle', 'rb'))
To simplify indexing later for plots, we create shortened keys. Additionally, we ca also separate the data based upon unique index set and thread numbers.
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_1k_omp1_params','_'):str_out[k]})
import pandas as pd
u_dat={}
threads_list=[16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = []
same_t.strum_metis_a = []
same_t.strum_scotch_bi = []
same_t.strum_metis_bi = []
same_t.strum_scotch_pr = []
same_t.strum_metis_pr = []
same_t.eig_llt = []
same_t.eig_ldlt = []
same_t.eig_bicgstab = []
for t in threads_list:
same_t.strum_metis_a.append(str_out_sm["omp%d_%s"%(t,u)][-2].rsplit(',')[-2].strip()[0:-1])
same_t.eig_ldlt.append(str_out_sm["omp%d_%s"%(t,u)][-3].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_bi.append(str_out_sm["omp%d_%s"%(t,u)][-4].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_pr.append(str_out_sm["omp%d_%s"%(t,u)][-5].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_a.append(str_out_sm["omp%d_%s"%(t,u)][-6].rsplit(',')[-2].strip()[0:-1])
same_t.eig_bicgstab.append(str_out_sm["omp%d_%s"%(t,u)][-7].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_bi.append(str_out_sm["omp%d_%s"%(t,u)][-8].rsplit(',')[-2].strip()[0:-1])
same_t.eig_llt.append(str_out_sm["omp%d_%s"%(t,u)][-9].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_pr.append(str_out_sm["omp%d_%s"%(t,u)][-10].rsplit(',')[-2].strip()[0:-1])
u_dat.update({u:same_t})
str_E_LLT = "EIG_LLT_%s"%u
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SCOTCH_A = "STRUM_SCOTCHA_%s"%u
str_S_METIS_A = "STRUM_METISA_%s"%u
str_S_SCOTCH_BI = "STRUM_SCOTCHBI_%s"%u
str_S_METIS_BI = "STRUM_METISBI_%s"%u
str_S_SCOTCH_PR = "STRUM_SCOTCHPR_%s"%u
str_S_METIS_PR = "STRUM_METISPR_%s"%u
df_list.append( pd.DataFrame({str_E_LLT:same_t.eig_llt, str_E_LDLT:same_t.eig_ldlt, str_E_BICGSTAB:same_t.eig_bicgstab, str_S_SCOTCH_A:same_t.strum_scotch_a, str_S_METIS_A:same_t.strum_metis_a, str_S_SCOTCH_BI:same_t.strum_scotch_bi, str_S_METIS_BI:same_t.strum_metis_bi, str_S_SCOTCH_PR:same_t.strum_scotch_pr, str_S_METIS_PR:same_t.strum_metis_pr }, index=threads_list).transpose() )
We now have a list of Pandas Dataframes, with which we can combine to a single entity. We can then proceed to plot the resulting columns, comparing both Eigen and STRUMPACK for each specific set of refined parameters.
perf_s = pd.concat(df_list).transpose()
perf_s.transpose()
—CLICK FOR OUTPUT—
| 16 | |
|---|---|
| EIG_BICGSTAB_Rxy-Deff-Eta- | 0.024 |
| EIG_LDLT_Rxy-Deff-Eta- | 2.119 |
| EIG_LLT_Rxy-Deff-Eta- | 1.994 |
| STRUM_METISA_Rxy-Deff-Eta- | 2.157 |
| STRUM_METISBI_Rxy-Deff-Eta- | 0.104 |
| STRUM_METISPR_Rxy-Deff-Eta- | 2.350 |
| STRUM_SCOTCHA_Rxy-Deff-Eta- | 0.220 |
| STRUM_SCOTCHBI_Rxy-Deff-Eta- | 0.111 |
| STRUM_SCOTCHPR_Rxy-Deff-Eta- | 0.187 |
| EIG_BICGSTAB_Bfactor- | 0.049 |
| EIG_LDLT_Bfactor- | 0.238 |
| EIG_LLT_Bfactor- | 0.232 |
| STRUM_METISA_Bfactor- | 1.831 |
| STRUM_METISBI_Bfactor- | 0.177 |
| STRUM_METISPR_Bfactor- | 2.001 |
| STRUM_SCOTCHA_Bfactor- | 0.447 |
| STRUM_SCOTCHBI_Bfactor- | 0.169 |
| STRUM_SCOTCHPR_Bfactor- | 0.422 |
| EIG_BICGSTAB_Bfactor-Deff-Eta- | 0.051 |
| EIG_LDLT_Bfactor-Deff-Eta- | 0.863 |
| EIG_LLT_Bfactor-Deff-Eta- | 0.835 |
| STRUM_METISA_Bfactor-Deff-Eta- | 2.025 |
| STRUM_METISBI_Bfactor-Deff-Eta- | 0.166 |
| STRUM_METISPR_Bfactor-Deff-Eta- | 2.094 |
| STRUM_SCOTCHA_Bfactor-Deff-Eta- | 0.466 |
| STRUM_SCOTCHBI_Bfactor-Deff-Eta- | 0.148 |
| STRUM_SCOTCHPR_Bfactor-Deff-Eta- | 0.426 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff-Eta- | 0.047 |
| EIG_LDLT_Rxy-Bfactor-Deff-Eta- | 4.073 |
| EIG_LLT_Rxy-Bfactor-Deff-Eta- | 3.903 |
| ... | ... |
| STRUM_SCOTCHA_Deff- | 0.215 |
| STRUM_SCOTCHBI_Deff- | 0.109 |
| STRUM_SCOTCHPR_Deff- | 0.192 |
| EIG_BICGSTAB_Rxy-Bfactor- | 0.048 |
| EIG_LDLT_Rxy-Bfactor- | 2.120 |
| EIG_LLT_Rxy-Bfactor- | 1.995 |
| STRUM_METISA_Rxy-Bfactor- | 2.191 |
| STRUM_METISBI_Rxy-Bfactor- | 0.163 |
| STRUM_METISPR_Rxy-Bfactor- | 2.391 |
| STRUM_SCOTCHA_Rxy-Bfactor- | 0.417 |
| STRUM_SCOTCHBI_Rxy-Bfactor- | 0.153 |
| STRUM_SCOTCHPR_Rxy-Bfactor- | 0.433 |
| EIG_BICGSTAB_Rxy-Bfactor-Eta- | 0.048 |
| EIG_LDLT_Rxy-Bfactor-Eta- | 2.119 |
| EIG_LLT_Rxy-Bfactor-Eta- | 2.023 |
| STRUM_METISA_Rxy-Bfactor-Eta- | 2.411 |
| STRUM_METISBI_Rxy-Bfactor-Eta- | 0.161 |
| STRUM_METISPR_Rxy-Bfactor-Eta- | 2.354 |
| STRUM_SCOTCHA_Rxy-Bfactor-Eta- | 0.401 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Eta- | 0.151 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Eta- | 0.434 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff- | 0.050 |
| EIG_LDLT_Rxy-Bfactor-Deff- | 4.098 |
| EIG_LLT_Rxy-Bfactor-Deff- | 3.883 |
| STRUM_METISA_Rxy-Bfactor-Deff- | 2.365 |
| STRUM_METISBI_Rxy-Bfactor-Deff- | 0.167 |
| STRUM_METISPR_Rxy-Bfactor-Deff- | 2.539 |
| STRUM_SCOTCHA_Rxy-Bfactor-Deff- | 0.435 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Deff- | 0.154 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Deff- | 0.450 |
135 rows × 1 columns
With the performance data in an easily accessible format, we can plot each respective data set for all solvers. Each bar corresponds to the parameters being refined, where the colour indicates the respective solver used, and the y-axis is the time to solve the system in seconds.
# define the figure size and grid layout properties
matplotlib.rcParams.update({'font.size': 22})
figsize = (20, 16)
cols = 4
fig3 = plt.figure(num=3, figsize=figsize)
ax3 = plt.axes()
ii = []
uu=[]
keys=['EIG_LLT','EIG_LDLT','STRUM_METISA','STRUM_SCOTCHA','STRUM_METISBI','STRUM_SCOTCHBI','STRUM_METISPR','STRUM_SCOTCHPR','EIG_BICGSTAB']
colours = ['#a6cee3','#1f78b4','#b2df8a','#33a02c','#fb9a99','#e31a1c','#fdbf6f','#ff7f00','#cab2d6']
key_col = {k:v for (k,v) in zip(keys,colours)}
hatch_pattern = [ '/' , '\\' , '|' , '-' , '+' , 'x' ,'o' ,'O' , '.', '*']
for i, u in enumerate(uniq_ref):
p_list=[]
ax3.set_ylabel('time [s]')
keys_df = [k+'_'+u for k in keys]
dfl=perf_s[keys_df]
m = zip(dfl.as_matrix().astype(np.float)[0], dfl.keys().tolist() )
m = sorted(m,key=lambda x: x[0])
for p in reversed(xrange(len(m))):
c_label = "_".join(m[p][1].split('_')[0:2])
p_list.append( plt.bar(i, m[p][0], label=m[p][1], color=key_col[c_label], edgecolor='k') )
ii.append(i)
uu.append(u)
ax3.set_yscale('log',basey=10)
plt.title('1k frames OMP_NUM_THREADS=16', fontsize=24)
plt.xticks(ii, uu)
plt.setp(ax3.get_xticklabels(), rotation=90, ha='left')
ax3.legend(p_list,keys,bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.,)
plt.savefig('omp_1kframes_allsolvers_omp16.pdf', pad_inches=10)
 |
So far we have examined the single threaded and OpenMP enabled solver performance. Strumpack also allows us to use the distributed MPI backend. We can set this up as follows using the ipyparallel backend, or invoke an mpirun call using the cell bash bash environment. For completeness, the ipyparallel environment can be set up as follows:
conda install ipyparallel
jupyter serverextension enable --py ipyparallel --user
jupyter nbextension install --py ipyparallel --user
jupyter nbextension enable --py ipyparallel --user
ipcluster start --profile=mpi -n 4
Take note of the controller client json file <HOME>/.ipython/profile_<PROFILE_NAME>/security/ipcontroller-client.json. This will be used to allow the notebook to connect to the ipyparallel cluster profile mpi. We can then return the handle of the cluster environment with:
import ipyparallel as ipp
#c = ipp.Client('/net/cci-filer2/raid1/home/mlxd/.ipython/profile_mpi/security/ipcontroller-client.json')
c = ipp.Client()
rc=c[:]
c.ids #Instance IDs
[0]
For our purposes it can be easier to call the solver using the ! bash environment, as was used for the OpenMP studies earlier. A simple test script can be performed as follows, which tests the listed solvers against the MPI backend.
!OMP_NUM_THREADS=1 mpirun -np 2 libtbx.python \
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/\
modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
14151 7075.5
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s
individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s
individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s
CPP -2
# Initializing STRUMPACK
# using 1 OpenMP thread
CPP -2
# using 2 MPI processes
# matching job: maximum matching with row and column scaling
# initial matrix:
# - number of unknowns = 14,151
# - number of nonzeros = 72,305
# nested dissection reordering:
# - Scotch reordering
# - strategy parameter = 8
# - number of separators = 4,075
# - number of levels = 68
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
# - nd time = 0.0810261
# - matching time = 0.0116782
# - symmetrization time = 0.00337219
# symbolic factorization:
# - nr of dense Frontal matrices = 4,075
# - nr of HSS Frontal matrices = 0
# - symb-factor time = 0.021524
# multifrontal factorization:
# - estimated memory usage (exact solver) = 7.70359 MB
# - factor time = 0.223322
# - factor nonzeros = 962,949
# - factor memory = 7.70359 MB
# - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08
# - flop rate = 1.11548 GFlop/s
# - factor memory/nonzeros = 100 % of multifrontal
# - maximum HSS rank = 0
# - HSS compression = false
# - relative compression tolerance = 0.01
# - absolute compression tolerance = 1e-08
# - normal(0,1) distribution with minstd_rand engine
# ----- FLOP BREAKDOWN ---------------------
# compression = 0
# random = 0
# ID = 0
# QR = 0
# ortho = 0
# reduce_samples = 0
# update_samples = 0
# extraction = 0
# sampling = 0
# CB_sample = 0
# sparse_sampling = 0
# ULV_factor = 0
# Schur = 0
# full_rank = 2.63271e+08
# --------------------------------------------
# total = 2.63271e+08
# --------------------------------------------
REFINEMENT it. 0 res = 15322 rel.res = 1 bw.error = 1
REFINEMENT it. 1 res = 1.1079e-11 rel.res = 7.23077e-16 bw.error = 1.25632e-15
# DIRECT/GMRES solve:
# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000
# - number of Krylov iterations = 1
# - solve time = 0.011821
# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06
# - flop rate = 0.209459 GFlop/s
# - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB
# - byte rate = 1.84722 GByte/s
# - solve arithmetic intensity = 0.113391 flop/byte
individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s
individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s
Exiting profiler
time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s, #calls: 1
TOTAL : CPU, 0.350s; elapsed, 0.359s
Strumpack solutions agree with Eigen
Exiting profiler
time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s, #calls: 1
time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s, #calls: 1
time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s, #calls: 1
time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s, #calls: 1
TOTAL : CPU, 0.790s; elapsed, 0.786s
We now use the same procedure of creating a solver file, write to disk, and call with the respective matrix arguments with OMP_NUM_THREADS and mpirun -n. For simplicity, we examine the OpenMP STRUM_SCOTCHA, EIGEN_LDLT and EIGEN_BICGSTAB, compared with the MPI-enabled STRUM_SCOTCHA (we may also examine the full range of algorithms as used in the previous example).
MPI_SOLVER='''
from __future__ import division
import mpi4py
mpi4py.rc.threads = True
mpi4py.rc.thread_level = "funneled"
from mpi4py import MPI
assert MPI.Is_initialized()
assert MPI.Query_thread() == MPI.THREAD_FUNNELED
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
from scitbx.matrix import sqr,col
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
import sys
import numpy as np
import scipy.sparse as sps
if rank==0:
A_mat = np.loadtxt(sys.argv[1],dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_vec = np.loadtxt(sys.argv[2])
n_rows = len(b_vec)
n_cols = n_rows
nnz = len(A_mat['vals'])
print n_rows, n_rows/size
#Convert the sparse CSR to flex doubles, then use them to solve using the implemented framework
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check if upper/lower triangular only, and generate full if so
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
import numpy as np
row_idx_split = np.array_split(np.arange(n_rows),size)
len_row_idx_split = flex.int( np.cumsum( np.append([0], [len(i) for i in row_idx_split]) ).tolist() )
P = Profiler("STRUMPACK_OMP")
A_indptr = flex.int( A_spS.indptr )
A_indices = flex.int( A_spS.indices )
A_data = flex.double( A_spS.data )
b = flex.double( b_vec )
res_strum_omp = ext_omp.strumpack_solver(
n_rows, n_cols,
A_indptr, A_indices,
A_data, b,
ext_omp.scotch,
ext_omp.auto
)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = ext_omp.eigen_solver(1, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = ext_omp.eigen_solver(2, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
else:
A_spS=None
row_idx_split=None
len_row_idx_split=None
b_vec=None
n_cols=None
if size>1:
#Broadcast data to each rank
A_spS = comm.bcast(A_spS, root=0)
row_idx_split = comm.bcast(row_idx_split, root=0)
len_row_idx_split = comm.bcast(len_row_idx_split, root=0)
b_vec = comm.bcast(b_vec, root=0)
n_cols = comm.bcast(n_cols, root=0)
#Take subset of data for each rank
A_row_offset = flex.int(A_spS[row_idx_split[rank],:].indptr)
A_col_offset = flex.int(A_spS[row_idx_split[rank],:].indices)
A_values = flex.double(A_spS[row_idx_split[rank],:].data)
b = flex.double(b_vec[row_idx_split[rank]])
P = Profiler("STRUMPACK_MPI_DIST_RANK=%d"%rank)
res_strum_mpi_local = ext_mpi.strumpack_mpi_dist_solver(len(row_idx_split[rank]), n_cols, comm, A_row_offset, A_col_offset, A_values, b, len_row_idx_split, ext_mpi.scotch, ext_mpi.auto)
strum_result_mpi_list = comm.gather(res_strum_mpi_local.x, root=0)
del P
if rank==0:
strum_result_mpi = flex.double()
for l in strum_result_mpi_list:
strum_result_mpi.extend(l)
#MPI.Finalize()
if rank==0:
strum_result_omp = res_strum_omp.x
eig_result_ldlt = res_eig_ldlt.x
eig_result_bicgstab = res_eig_bicgstab.x
for ii in xrange(len(strum_result_omp)):
assert approx_equal( strum_result_omp[ii], eig_result_ldlt[ii] )
assert approx_equal( strum_result_omp[ii], eig_result_bicgstab[ii] )
if size>1:
assert approx_equal( strum_result_omp[ii], strum_result_mpi[ii] )
print "Strumpack solutions agree with Eigen"
'''
MPI_SOLVER_FILE = open("MPI_SOLVER.py", "w")
MPI_SOLVER_FILE.write(MPI_SOLVER)
MPI_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We intend to examine performance while varying both OMP_NUM_THREADS and the MPI process number to determine an optimal set of parameters for the distributed backend. In the following example we test between 1 to 16 OpenMP threads per process, and from 1 to 16 MPI processes, assuming a total of threads*processes <= 32 cores. The resulting data set will be a scalar field of time as a function of both variables, which we can plot as a heatmap. We can begin by running the analysis on a 1000 image dataset.
str_out={}
import os
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "10k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for procs in mpi_proc_list:
for threads in threads_list:
if threads*procs <= 32:
print "mpirun -n %d"%procs
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} mpirun -n {procs} libtbx.python ./MPI_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting with log: %s"%val)
key = 'mpi' + str(procs) + 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.130s; elapsed, 31.154s', 'individual call time for EIGEN_LDLT: CPU, 37.630s; elapsed, 37.656s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.130s; elapsed, 31.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.630s; elapsed, 37.656s, #calls: 1', 'TOTAL : CPU, 68.880s; elapsed, 68.927s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.010s; elapsed, 17.742s', 'individual call time for EIGEN_LDLT: CPU, 37.800s; elapsed, 37.629s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.010s; elapsed, 17.742s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.800s; elapsed, 37.629s, #calls: 1', 'TOTAL : CPU, 72.980s; elapsed, 55.492s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 58.500s; elapsed, 14.890s', 'individual call time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.722s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 58.500s; elapsed, 14.890s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.722s, #calls: 1', 'TOTAL : CPU, 96.920s; elapsed, 52.711s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 114.020s; elapsed, 14.496s', 'individual call time for EIGEN_LDLT: CPU, 38.940s; elapsed, 37.725s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.094s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 114.020s; elapsed, 14.496s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.940s; elapsed, 37.725s, #calls: 1', 'TOTAL : CPU, 153.230s; elapsed, 52.315s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 305.110s; elapsed, 19.364s', 'individual call time for EIGEN_LDLT: CPU, 40.400s; elapsed, 37.744s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.098s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 305.110s; elapsed, 19.364s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.098s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.400s; elapsed, 37.744s, #calls: 1', 'TOTAL : CPU, 345.950s; elapsed, 57.206s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.410s; elapsed, 31.429s', 'individual call time for EIGEN_LDLT: CPU, 37.660s; elapsed, 37.677s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.423636', '# - matching time = 0.0671489', '# - symmetrization time = 0.051945', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084671', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 30.2248', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 9.35496e+10', '# - flop rate = 3.09512 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73537e-11\trel.res = 1.86806e-15\tbw.error = 5.7415e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.132668', '# - total flops = 6.72147e+07, min = 165052, max = 6.70497e+07', '# - flop rate = 0.506638 GFlop/s', '# - bytes moved = 408.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.07629 GByte/s', '# - solve arithmetic intensity = 0.164691 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.050s; elapsed, 31.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.070s; elapsed, 31.079s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.050s; elapsed, 31.068s, #calls: 1', 'TOTAL : CPU, 31.050s; elapsed, 31.068s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.410s; elapsed, 31.429s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.660s; elapsed, 37.677s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.070s; elapsed, 31.079s, #calls: 1', 'TOTAL : CPU, 100.260s; elapsed, 100.301s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.808s', 'individual call time for EIGEN_LDLT: CPU, 37.740s; elapsed, 37.569s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.420951', '# - matching time = 0.068316', '# - symmetrization time = 0.051847', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0771451', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.5968', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 5.63661 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.80099e-11\trel.res = 1.88391e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.116206', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.578566 GFlop/s', '# - bytes moved = 408.883 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.5186 GByte/s', '# - solve arithmetic intensity = 0.164431 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.470s; elapsed, 17.417s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.180s; elapsed, 17.429s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.470s; elapsed, 17.417s, #calls: 1', 'TOTAL : CPU, 34.470s; elapsed, 17.417s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.808s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.740s; elapsed, 37.569s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.180s; elapsed, 17.429s, #calls: 1', 'TOTAL : CPU, 91.170s; elapsed, 72.911s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 63.980s; elapsed, 16.291s', 'individual call time for EIGEN_LDLT: CPU, 39.450s; elapsed, 38.939s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.106s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.463055', '# - matching time = 0.0712121', '# - symmetrization time = 0.0550501', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0739958', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.3268', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 5.72984 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79581e-11\trel.res = 1.88266e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.197363', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.340656 GFlop/s', '# - bytes moved = 408.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.07183 GByte/s', '# - solve arithmetic intensity = 0.164423 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.860s; elapsed, 17.288s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.790s; elapsed, 17.298s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.860s; elapsed, 17.288s, #calls: 1', 'TOTAL : CPU, 67.860s; elapsed, 17.288s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 63.980s; elapsed, 16.291s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.106s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.450s; elapsed, 38.939s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.790s; elapsed, 17.298s, #calls: 1', 'TOTAL : CPU, 123.440s; elapsed, 72.635s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 120.850s; elapsed, 15.440s', 'individual call time for EIGEN_LDLT: CPU, 59.540s; elapsed, 58.326s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.44408', '# - matching time = 0.087352', '# - symmetrization time = 0.0665729', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0964291', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 14.9592', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 42535, max = 9.35498e+10', '# - flop rate = 6.25367 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79316e-11\trel.res = 1.88202e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198861', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.338089 GFlop/s', '# - bytes moved = 408.94 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05641 GByte/s', '# - solve arithmetic intensity = 0.164408 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.210s; elapsed, 15.962s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.160s; elapsed, 15.972s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.210s; elapsed, 15.962s, #calls: 1', 'TOTAL : CPU, 125.210s; elapsed, 15.962s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 120.850s; elapsed, 15.440s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 59.540s; elapsed, 58.326s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.160s; elapsed, 15.972s, #calls: 1', 'TOTAL : CPU, 202.850s; elapsed, 89.851s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 250.560s; elapsed, 15.947s', 'individual call time for EIGEN_LDLT: CPU, 40.520s; elapsed, 37.817s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.094s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7CPP -2', ' = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.445719', '# - matching time = 0.0899839', '# - symmetrization time = 0.0637009', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0908601', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.9512', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35508e+10, min = 43208, max = 9.35507e+10', '# - flop rate = 5.51884 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.68634e-11\trel.res = 1.85622e-15\tbw.error = 5.74638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198701', '# - total flops = 6.72375e+07, min = 165052, max = 6.70724e+07', '# - flop rate = 0.338385 GFlop/s', '# - bytes moved = 409.091 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05883 GByte/s', '# - solve arithmetic intensity = 0.164358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 281.170s; elapsed, 17.946s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.240s; elapsed, 17.957s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 281.170s; elapsed, 17.946s, #calls: 1', 'TOTAL : CPU, 281.170s; elapsed, 17.946s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 250.560s; elapsed, 15.947s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.520s; elapsed, 37.817s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.240s; elapsed, 17.957s, #calls: 1', 'TOTAL : CPU, 322.740s; elapsed, 71.815s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 37.500s; elapsed, 37.561s', 'individual call time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.984s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.122s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.452391', '# - matching time = 0.070359', '# - symmetrization time = 0.0538969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0570049', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 17.0081', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 5.5003 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73296e-11\trel.res = 1.86748e-15\tbw.error = 5.73834e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.10983', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.645948 GFlop/s', '# - bytes moved = 217.757 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98267 GByte/s', '# - solve arithmetic intensity = 0.325796 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.800s; elapsed, 17.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.780s; elapsed, 17.793s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.780s; elapsed, 17.807s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.810s; elapsed, 17.814s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.780s; elapsed, 17.807s, #calls: 1', 'TOTAL : CPU, 17.780s; elapsed, 17.807s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.780s; elapsed, 17.793s, #calls: 1', 'TOTAL : CPU, 17.780s; elapsed, 17.793s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.800s; elapsed, 17.815s, #calls: 1', 'TOTAL : CPU, 17.800s; elapsed, 17.815s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 37.500s; elapsed, 37.561s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.122s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.984s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.810s; elapsed, 17.814s, #calls: 1', 'TOTAL : CPU, 93.380s; elapsed, 93.481s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.260s; elapsed, 18.367s', 'individual call time for EIGEN_LDLT: CPU, 38.970s; elapsed, 38.812s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.109s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.445381', '# - matching time = 0.073508', '# - symmetrization time = 0.0528131', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.059128', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 17.3075', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 5.40515 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71926e-11\trel.res = 1.86417e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.107487', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.660028 GFlop/s', '# - bytes moved = 217.769 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.02601 GByte/s', '# - solve arithmetic intensity = 0.325778 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 18.116s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.570s; elapsed, 18.094s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.550s; elapsed, 18.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.810s; elapsed, 18.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.550s; elapsed, 18.109s, #calls: 1', 'TOTAL : CPU, 35.550s; elapsed, 18.109s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 18.116s, #calls: 1', 'TOTAL : CPU, 25.390s; elapsed, 18.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.570s; elapsed, 18.094s, #calls: 1', 'TOTAL : CPU, 27.570s; elapsed, 18.094s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.260s; elapsed, 18.367s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.970s; elapsed, 38.812s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.810s; elapsed, 18.114s, #calls: 1', 'TOTAL : CPU, 94.180s; elapsed, 75.402s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 69.500s; elapsed, 17.692s', 'individual call time for EIGEN_LDLT: CPU, 38.230s; elapsed, 37.698s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.093s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.460802', '# - matching time = 0.0799248', '# - symmetrization time = 0.0598788', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0779431', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.5414', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 5.65547 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.7193e-11\trel.res = 1.86418e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.112011', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.633368 GFlop/s', '# - bytes moved = 217.784 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94431 GByte/s', '# - solve arithmetic intensity = 0.325755 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.940s; elapsed, 17.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 48.640s; elapsed, 17.384s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.640s; elapsed, 17.396s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.600s; elapsed, 17.403s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.640s; elapsed, 17.396s, #calls: 1', 'TOTAL : CPU, 67.640s; elapsed, 17.396s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 48.640s; elapsed, 17.384s, #calls: 1', 'TOTAL : CPU, 48.640s; elapsed, 17.384s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.940s; elapsed, 17.404s, #calls: 1', 'TOTAL : CPU, 37.940s; elapsed, 17.404s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 69.500s; elapsed, 17.692s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.230s; elapsed, 37.698s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.600s; elapsed, 17.403s, #calls: 1', 'TOTAL : CPU, 127.500s; elapsed, 72.886s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 111.720s; elapsed, 14.239s', 'individual call time for EIGEN_LDLT: CPU, 40.080s; elapsed, 38.861s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.094s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.491927', '# - matching time = 0.0973151', '# - symmetrization time = 0.0595419', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0807052', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.3578', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 42535, max = 7.04191e+10', '# - flop rate = 6.09135 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71813e-11\trel.res = 1.8639e-15\tbw.error = 5.74928e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.136156', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.521053 GFlop/s', '# - bytes moved = 217.815 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59975 GByte/s', '# - solve arithmetic intensity = 0.325709 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.840s; elapsed, 16.297s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 91.810s; elapsed, 16.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.580s; elapsed, 16.290s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.650s; elapsed, 16.296s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.580s; elapsed, 16.290s, #calls: 1', 'TOTAL : CPU, 125.580s; elapsed, 16.290s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 91.810s; elapsed, 16.276s, #calls: 1', 'TOTAL : CPU, 91.810s; elapsed, 16.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.840s; elapsed, 16.297s, #calls: 1', 'TOTAL : CPU, 60.840s; elapsed, 16.297s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 111.720s; elapsed, 14.239s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.080s; elapsed, 38.861s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.650s; elapsed, 16.296s, #calls: 1', 'TOTAL : CPU, 173.710s; elapsed, 69.489s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.540s; elapsed, 32.559s', 'individual call time for EIGEN_LDLT: CPU, 38.210s; elapsed, 38.240s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.443409', '# - matching time = 0.0673399', '# - symmetrization time = 0.0424738', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0652878', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 17.2776', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 5.50197 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01663e-10\trel.res = 2.45512e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.106128', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.765183 GFlop/s', '# - bytes moved = 211.022 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98838 GByte/s', '# - solve arithmetic intensity = 0.384828 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.020s; elapsed, 18.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.030s; elapsed, 18.064s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.040s; elapsed, 18.059s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.040s; elapsed, 18.056s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.990s; elapsed, 18.046s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.030s; elapsed, 18.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.030s; elapsed, 18.058s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.070s; elapsed, 18.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.030s; elapsed, 18.064s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.064s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.020s; elapsed, 18.062s, #calls: 1', 'TOTAL : CPU, 18.020s; elapsed, 18.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.990s; elapsed, 18.046s, #calls: 1', 'TOTAL : CPU, 17.990s; elapsed, 18.046s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.030s; elapsed, 18.057s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.040s; elapsed, 18.056s, #calls: 1', 'TOTAL : CPU, 18.040s; elapsed, 18.056s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.040s; elapsed, 18.059s, #calls: 1', 'TOTAL : CPU, 18.040s; elapsed, 18.059s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.030s; elapsed, 18.058s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.058s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.540s; elapsed, 32.559s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.210s; elapsed, 38.240s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.070s; elapsed, 18.065s, #calls: 1', 'TOTAL : CPU, 88.940s; elapsed, 88.984s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.480s; elapsed, 20.502s', 'individual call time for EIGEN_LDLT: CPU, 37.870s; elapsed, 37.694s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.108s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.465065', '# - matching time = 0.079277', '# - symmetrization time = 0.039073', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.08726', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 14.0964', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 6.74364 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01627e-10\trel.res = 2.45425e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0849791', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.955615 GFlop/s', '# - bytes moved = 211.031 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48333 GByte/s', '# - solve arithmetic intensity = 0.384812 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.130s; elapsed, 14.914s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.200s; elapsed, 14.914s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.150s; elapsed, 14.908s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.870s; elapsed, 14.911s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 14.909s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.760s; elapsed, 14.910s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 23.480s; elapsed, 14.899s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.620s; elapsed, 14.917s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.870s; elapsed, 14.911s, #calls: 1', 'TOTAL : CPU, 28.870s; elapsed, 14.911s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 14.909s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 14.909s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 23.480s; elapsed, 14.899s, #calls: 1', 'TOTAL : CPU, 23.480s; elapsed, 14.899s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.200s; elapsed, 14.914s, #calls: 1', 'TOTAL : CPU, 19.200s; elapsed, 14.914s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.150s; elapsed, 14.908s, #calls: 1', 'TOTAL : CPU, 17.150s; elapsed, 14.908s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.130s; elapsed, 14.914s, #calls: 1', 'TOTAL : CPU, 21.130s; elapsed, 14.914s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.760s; elapsed, 14.910s, #calls: 1', 'TOTAL : CPU, 16.760s; elapsed, 14.910s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.480s; elapsed, 20.502s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.108s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.870s; elapsed, 37.694s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.620s; elapsed, 14.917s, #calls: 1', 'TOTAL : CPU, 94.120s; elapsed, 73.220s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 59.340s; elapsed, 15.228s', 'individual call time for EIGEN_LDLT: CPU, 40.550s; elapsed, 40.039s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.521897', '# - matching time = 0.0839229', '# - symmetrization time = 0.0437839', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.072571', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 18.7376', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 5.07328 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0162e-10\trel.res = 2.45408e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.102959', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.788733 GFlop/s', '# - bytes moved = 211.042 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04976 GByte/s', '# - solve arithmetic intensity = 0.384792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 26.920s; elapsed, 19.628s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 38.880s; elapsed, 19.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.440s; elapsed, 19.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 51.330s; elapsed, 19.616s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.080s; elapsed, 19.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 75.560s; elapsed, 19.629s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 19.632s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.770s; elapsed, 19.636s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 26.920s; elapsed, 19.628s, #calls: 1', 'TOTAL : CPU, 26.920s; elapsed, 19.628s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.080s; elapsed, 19.630s, #calls: 1', 'TOTAL : CPU, 26.080s; elapsed, 19.630s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 51.330s; elapsed, 19.616s, #calls: 1', 'TOTAL : CPU, 51.330s; elapsed, 19.616s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 38.880s; elapsed, 19.634s, #calls: 1', 'TOTAL : CPU, 38.880s; elapsed, 19.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 75.560s; elapsed, 19.629s, #calls: 1', 'TOTAL : CPU, 75.560s; elapsed, 19.629s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 19.632s, #calls: 1', 'TOTAL : CPU, 25.390s; elapsed, 19.632s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.440s; elapsed, 19.635s, #calls: 1', 'TOTAL : CPU, 30.440s; elapsed, 19.635s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 59.340s; elapsed, 15.228s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.550s; elapsed, 40.039s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.770s; elapsed, 19.636s, #calls: 1', 'TOTAL : CPU, 121.880s; elapsed, 75.021s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.680s; elapsed, 32.706s', 'individual call time for EIGEN_LDLT: CPU, 39.000s; elapsed, 39.039s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = 0.449232', '# - matching time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.070698', '# - symmetrization time = 0.0213499', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.066813', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 17.1123', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 5.55511 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56982e-11\trel.res = 2.31108e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0819831', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.23334 GFlop/s', '# - bytes moved = 203.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.4827 GByte/s', '# - solve arithmetic intensity = 0.496775 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.760s; elapsed, 17.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.820s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.830s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.750s; elapsed, 17.853s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.840s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.750s; elapsed, 17.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.740s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.750s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.820s; elapsed, 17.851s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.750s; elapsed, 17.848s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 17.851s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.830s; elapsed, 17.852s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.750s; elapsed, 17.849s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.730s; elapsed, 17.848s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.720s; elapsed, 17.852s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.850s; elapsed, 17.859s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.750s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.750s; elapsed, 17.848s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.848s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.750s; elapsed, 17.853s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.853s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.840s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.840s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.820s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.820s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.760s; elapsed, 17.855s, #calls: 1', 'TOTAL : CPU, 17.760s; elapsed, 17.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.740s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.740s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.830s; elapsed, 17.852s, #calls: 1', 'TOTAL : CPU, 17.830s; elapsed, 17.852s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.750s; elapsed, 17.855s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.750s; elapsed, 17.849s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.849s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.720s; elapsed, 17.852s, #calls: 1', 'TOTAL : CPU, 17.720s; elapsed, 17.852s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.830s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.830s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 17.851s, #calls: 1', 'TOTAL : CPU, 17.730s; elapsed, 17.851s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.730s; elapsed, 17.848s, #calls: 1', 'TOTAL : CPU, 17.730s; elapsed, 17.848s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.820s; elapsed, 17.851s, #calls: 1', 'TOTAL : CPU, 17.820s; elapsed, 17.851s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.680s; elapsed, 32.706s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.000s; elapsed, 39.039s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.850s; elapsed, 17.859s, #calls: 1', 'TOTAL : CPU, 89.650s; elapsed, 89.723s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.100s; elapsed, 19.255s', 'individual call time for EIGEN_LDLT: CPU, 38.600s; elapsed, 38.444s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.506749', '# - matching time = 0.079515', '# - symmetrization time = 0.0236838', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.081279', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 18.2385', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 5.21211 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56833e-11\trel.res = 2.31072e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079212', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.27649 GFlop/s', '# - bytes moved = 203.544 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56961 GByte/s', '# - solve arithmetic intensity = 0.496763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.820s; elapsed, 19.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.580s; elapsed, 19.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.380s; elapsed, 19.069s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 21.140s; elapsed, 19.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 25.440s; elapsed, 19.073s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 23.180s; elapsed, 19.071s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 23.760s; elapsed, 19.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 21.410s; elapsed, 19.065s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.780s; elapsed, 19.069s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.660s; elapsed, 19.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.700s; elapsed, 19.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.170s; elapsed, 19.069s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 21.620s; elapsed, 19.067s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 21.190s; elapsed, 19.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 28.200s; elapsed, 19.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.750s; elapsed, 19.075s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.820s; elapsed, 19.068s, #calls: 1', 'TOTAL : CPU, 20.820s; elapsed, 19.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.780s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 20.780s; elapsed, 19.069s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.380s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 21.380s; elapsed, 19.069s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 21.410s; elapsed, 19.065s, #calls: 1', 'TOTAL : CPU, 21.410s; elapsed, 19.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 23.180s; elapsed, 19.071s, #calls: 1', 'TOTAL : CPU, 23.180s; elapsed, 19.071s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 25.440s; elapsed, 19.073s, #calls: 1', 'TOTAL : CPU, 25.440s; elapsed, 19.073s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.660s; elapsed, 19.068s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.580s; elapsed, 19.070s, #calls: 1', 'TOTAL : CPU, 21.580s; elapsed, 19.070s', 'TOTAL : CPU, 20.660s; elapsed, 19.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.170s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 37.170s; elapsed, 19.069s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 21.140s; elapsed, 19.070s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 21.140s; elapsed, 19.070s', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 28.200s; elapsed, 19.062s, #calls: 1', 'TOTAL : CPU, 28.200s; elapsed, 19.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 23.760s; elapsed, 19.070s, #calls: 1', 'TOTAL : CPU, 23.760s; elapsed, 19.070s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 21.190s; elapsed, 19.068s, #calls: 1', 'TOTAL : CPU, 21.190s; elapsed, 19.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 21.620s; elapsed, 19.067s, #calls: 1', 'TOTAL : CPU, 21.620s; elapsed, 19.067s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.700s; elapsed, 19.061s, #calls: 1', 'TOTAL : CPU, 21.700s; elapsed, 19.061s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.100s; elapsed, 19.255s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.600s; elapsed, 38.444s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.750s; elapsed, 19.075s, #calls: 1', 'TOTAL : CPU, 96.620s; elapsed, 76.893s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 141.730s; elapsed, 141.848s', 'individual call time for EIGEN_LDLT: CPU, 292.340s; elapsed, 292.520s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.279s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 141.730s; elapsed, 141.848s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.279s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.340s; elapsed, 292.520s, #calls: 1', 'TOTAL : CPU, 434.360s; elapsed, 434.647s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 164.130s; elapsed, 83.129s', 'individual call time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 164.130s; elapsed, 83.129s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.863s, #calls: 1', 'TOTAL : CPU, 457.310s; elapsed, 376.231s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 246.360s; elapsed, 63.096s', 'individual call time for EIGEN_LDLT: CPU, 293.190s; elapsed, 292.847s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 246.360s; elapsed, 63.096s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.190s; elapsed, 292.847s, #calls: 1', 'TOTAL : CPU, 540.000s; elapsed, 356.183s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 538.270s; elapsed, 68.635s', 'individual call time for EIGEN_LDLT: CPU, 293.590s; elapsed, 293.120s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.178s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 538.270s; elapsed, 68.635s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.178s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.590s; elapsed, 293.120s, #calls: 1', 'TOTAL : CPU, 832.410s; elapsed, 361.933s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1020.030s; elapsed, 64.684s', 'individual call time for EIGEN_LDLT: CPU, 295.470s; elapsed, 293.099s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.177s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1020.030s; elapsed, 64.684s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT: CPU, 295.470s; elapsed, 293.099s, #calls: 1', 'TOTAL : CPU, 1316.420s; elapsed, 357.961s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 144.920s; elapsed, 145.000s', 'individual call time for EIGEN_LDLT: CPU, 293.070s; elapsed, 293.189s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.268s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.563893', '# - matching time = 0.121963', '# - symmetrization time = 0.1143', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.173911', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 83.9234', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.57346 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12487e-10\trel.res = 2.68831e-15\tbw.error = 6.87881e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.193389', '# - total flops = 2.36268e+08, min = 4.69568e+07, max = 1.89311e+08', '# - flop rate = 1.22172 GFlop/s', '# - bytes moved = 525.018 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71483 GByte/s', '# - solve arithmetic intensity = 0.450019 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.030s; elapsed, 85.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.090s; elapsed, 85.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.030s; elapsed, 85.109s, #calls: 1', 'TOTAL : CPU, 85.030s; elapsed, 85.109s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 144.920s; elapsed, 145.000s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.268s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.070s; elapsed, 293.189s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.090s; elapsed, 85.129s, #calls: 1', 'TOTAL : CPU, 523.340s; elapsed, 523.585s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 156.990s; elapsed, 78.931s', 'individual call time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.806s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.242s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.593915', '# - matching time = 0.124004', '# - symmetrization time = 0.118403', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.190537', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 82.3973', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.73226 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12473e-10\trel.res = 2.68797e-15\tbw.error = 6.87847e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.332306', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.711141 GFlop/s', '# - bytes moved = 526.314 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.58382 GByte/s', '# - solve arithmetic intensity = 0.449003 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 146.320s; elapsed, 83.786s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 165.160s; elapsed, 83.810s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 146.320s; elapsed, 83.786s, #calls: 1', 'TOTAL : CPU, 146.320s; elapsed, 83.786s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 156.990s; elapsed, 78.931s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.242s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.806s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 165.160s; elapsed, 83.810s, #calls: 1', 'TOTAL : CPU, 615.330s; elapsed, 455.788s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 326.460s; elapsed, 83.440s', 'individual call time for EIGEN_LDLT: CPU, 301.500s; elapsed, 301.186s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.233s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.56676', '# - matching time = 0.132276', '# - symmetrization time = 0.122117', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.196164', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 76.0822', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 9.45707 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12402e-10\trel.res = 2.68628e-15\tbw.error = 6.87847e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.201955', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 1.17015 GFlop/s', '# - bytes moved = 526.329 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.60617 GByte/s', '# - solve arithmetic intensity = 0.44899 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 229.530s; elapsed, 77.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 299.970s; elapsed, 77.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 229.530s; elapsed, 77.330s, #calls: 1', 'TOTAL : CPU, 229.530s; elapsed, 77.330s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 326.460s; elapsed, 83.440s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.233s, #calls: 1', 'time for EIGEN_LDLT: CPU, 301.500s; elapsed, 301.186s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 299.970s; elapsed, 77.347s, #calls: 1', 'TOTAL : CPU, 928.430s; elapsed, 462.206s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 656.220s; elapsed, 83.746s', 'individual call time for EIGEN_LDLT: CPU, 350.840s; elapsed, 350.455s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.204s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.601008', '# - matching time = 0.149213', '# - symmetrization time = 0.134562', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.236634', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 88.7679', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.10557 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12404e-10\trel.res = 2.68632e-15\tbw.error = 6.87887e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.266323', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.88733 GFlop/s', '# - bytes moved = 526.362 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.9764 GByte/s', '# - solve arithmetic intensity = 0.448962 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 400.040s; elapsed, 90.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 704.490s; elapsed, 90.196s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 400.040s; elapsed, 90.179s, #calls: 1', 'TOTAL : CPU, 400.040s; elapsed, 90.179s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 656.220s; elapsed, 83.746s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.204s, #calls: 1', 'time for EIGEN_LDLT: CPU, 350.840s; elapsed, 350.455s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 704.490s; elapsed, 90.196s, #calls: 1', 'TOTAL : CPU, 1712.150s; elapsed, 524.602s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1449.460s; elapsed, 91.683s', 'individual call time for EIGEN_LDLT: CPU, 302.120s; elapsed, 299.595s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.175s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.608151', '# - matching time = 0.156646', '# - symmetrization time = 0.18053', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.229522', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 100.584', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 7.15334 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12389e-10\trel.res = 2.68597e-15\tbw.error = 6.87887e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.391923', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.602967 GFlop/s', '# - bytes moved = 526.428 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.34319 GByte/s', '# - solve arithmetic intensity = 0.448906 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 996.030s; elapsed, 102.181s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1587.170s; elapsed, 102.199s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 996.030s; elapsed, 102.181s, #calls: 1', 'TOTAL : CPU, 996.030s; elapsed, 102.181s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1449.460s; elapsed, 91.683s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.175s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.120s; elapsed, 299.595s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1587.170s; elapsed, 102.199s, #calls: 1', 'TOTAL : CPU, 3339.670s; elapsed, 493.651s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 147.950s; elapsed, 148.084s', 'individual call time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.364s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.292s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', 'CPP -2', 'CPP -2', '# using 1 OpenMP thread', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.607033', '# - matching time = 0.134257', '# - symmetrization time = 0.0776892', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.159855', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 69.9338', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16766e+08, max = 5.51962e+11', '# - flop rate = 10.2248 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.75852e-11\trel.res = 2.09319e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.241741', '# - total flops = 2.50759e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 1.0373 GFlop/s', '# - bytes moved = 456.661 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.88905 GByte/s', '# - solve arithmetic intensity = 0.549114 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 71.150s; elapsed, 71.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 71.140s; elapsed, 71.175s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 71.020s; elapsed, 71.168s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 71.110s; elapsed, 71.188s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 71.150s; elapsed, 71.179s, #calls: 1', 'TOTAL : CPU, 71.150s; elapsed, 71.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 71.140s; elapsed, 71.175s, #calls: 1', 'TOTAL : CPU, 71.140s; elapsed, 71.175s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 71.020s; elapsed, 71.168s, #calls: 1', 'TOTAL : CPU, 71.020s; elapsed, 71.168s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 147.950s; elapsed, 148.084s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.292s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.364s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 71.110s; elapsed, 71.188s, #calls: 1', 'TOTAL : CPU, 521.490s; elapsed, 521.927s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 188.090s; elapsed, 95.490s', 'individual call time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.181s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.244s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.593759', '# - matching time = 0.136821', '# - symmetrization time = 0.078804', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.169685', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 86.2361', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16766e+08, max = 5.51962e+11', '# - flop rate = 8.29186 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40934e-11\trel.res = 2.00974e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.220926', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 1.13509 GFlop/s', '# - bytes moved = 457.114 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06908 GByte/s', '# - solve arithmetic intensity = 0.548595 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 97.270s; elapsed, 87.456s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 117.890s; elapsed, 87.449s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 127.830s; elapsed, 87.458s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 170.670s; elapsed, 87.472s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 117.890s; elapsed, 87.449s, #calls: 1', 'TOTAL : CPU, 117.890s; elapsed, 87.449s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 97.270s; elapsed, 87.456s, #calls: 1', 'TOTAL : CPU, 97.270s; elapsed, 87.456s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 127.830s; elapsed, 87.458s, #calls: 1', 'TOTAL : CPU, 127.830s; elapsed, 87.458s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 188.090s; elapsed, 95.490s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.244s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.181s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 170.670s; elapsed, 87.472s, #calls: 1', 'TOTAL : CPU, 661.260s; elapsed, 485.387s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 254.190s; elapsed, 64.677s', 'individual call time for EIGEN_LDLT: CPU, 304.590s; elapsed, 304.241s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.215s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.598257', '# - matching time = 0.135279', '# - symmetrization time = 0.0794539', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.170847', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 82.2278', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16767e+08, max = 5.51962e+11', '# - flop rate = 8.69606 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40844e-11\trel.res = 2.00952e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198475', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 1.26349 GFlop/s', '# - bytes moved = 457.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.30319 GByte/s', '# - solve arithmetic intensity = 0.54858 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 111.970s; elapsed, 83.431s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 201.980s; elapsed, 83.427s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 188.080s; elapsed, 83.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 323.740s; elapsed, 83.442s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 201.980s; elapsed, 83.427s, #calls: 1', 'TOTAL : CPU, 201.980s; elapsed, 83.427s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 111.970s; elapsed, 83.431s, #calls: 1', 'TOTAL : CPU, 111.970s; elapsed, 83.431s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 188.080s; elapsed, 83.423s, #calls: 1', 'TOTAL : CPU, 188.080s; elapsed, 83.423s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 254.190s; elapsed, 64.677s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.215s, #calls: 1', 'time for EIGEN_LDLT: CPU, 304.590s; elapsed, 304.241s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 323.740s; elapsed, 83.442s, #calls: 1', 'TOTAL : CPU, 882.990s; elapsed, 452.575s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 703.410s; elapsed, 88.912s', 'individual call time for EIGEN_LDLT: CPU, 304.160s; elapsed, 303.644s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.197s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.610624', '# - matching time = 0.153965', '# - symmetrization time = 0.0895329', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.198788', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 69.6335', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16767e+08, max = 5.51962e+11', '# - flop rate = 10.2689 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40817e-11\trel.res = 2.00946e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.260917', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 0.961111 GFlop/s', '# - bytes moved = 457.151 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.75209 GByte/s', '# - solve arithmetic intensity = 0.54855 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.120s; elapsed, 70.975s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 369.910s; elapsed, 70.971s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.540s; elapsed, 70.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 541.220s; elapsed, 70.987s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 369.910s; elapsed, 70.971s, #calls: 1', 'TOTAL : CPU, 369.910s; elapsed, 70.971s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.120s; elapsed, 70.975s, #calls: 1', 'TOTAL : CPU, 128.120s; elapsed, 70.975s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.540s; elapsed, 70.966s, #calls: 1', 'TOTAL : CPU, 310.540s; elapsed, 70.966s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 703.410s; elapsed, 88.912s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.197s, #calls: 1', 'time for EIGEN_LDLT: CPU, 304.160s; elapsed, 303.644s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 541.220s; elapsed, 70.987s, #calls: 1', 'TOTAL : CPU, 1549.420s; elapsed, 463.740s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 151.980s; elapsed, 152.089s', 'individual call time for EIGEN_LDLT: CPU, 306.960s; elapsed, 307.079s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.296s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.598349', '# - matching time = 0.133743', '# - symmetrization time = 0.0565939', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.157678', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 67.1467', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84773e+07, max = 5.53144e+11', '# - flop rate = 10.6931 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.52645e-11\trel.res = 1.79874e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.147024', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 1.94791 GFlop/s', '# - bytes moved = 432.794 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.94369 GByte/s', '# - solve arithmetic intensity = 0.661723 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.220s; elapsed, 68.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 68.200s; elapsed, 68.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 68.190s; elapsed, 68.259s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.200s; elapsed, 68.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.220s; elapsed, 68.263s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.030s; elapsed, 68.263s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 68.180s; elapsed, 68.260s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.230s; elapsed, 68.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 68.200s; elapsed, 68.257s, #calls: 1', 'TOTAL : CPU, 68.200s; elapsed, 68.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 68.190s; elapsed, 68.259s, #calls: 1', 'TOTAL : CPU, 68.190s; elapsed, 68.259s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.200s; elapsed, 68.262s, #calls: 1', 'TOTAL : CPU, 68.200s; elapsed, 68.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 68.180s; elapsed, 68.260s, #calls: 1', 'TOTAL : CPU, 68.180s; elapsed, 68.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.220s; elapsed, 68.262s, #calls: 1', 'TOTAL : CPU, 68.220s; elapsed, 68.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.030s; elapsed, 68.263s, #calls: 1', 'TOTAL : CPU, 68.030s; elapsed, 68.263s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.220s; elapsed, 68.263s, #calls: 1', 'TOTAL : CPU, 68.220s; elapsed, 68.263s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 151.980s; elapsed, 152.089s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.296s, #calls: 1', 'time for EIGEN_LDLT: CPU, 306.960s; elapsed, 307.079s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.230s; elapsed, 68.270s, #calls: 1', 'TOTAL : CPU, 527.460s; elapsed, 527.734s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 237.010s; elapsed, 119.786s', 'individual call time for EIGEN_LDLT: CPU, 334.340s; elapsed, 334.452s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.283s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.697687', '# - matching time = 0.156106', '# - symmetrization time = 0.065562', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.181438', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 60.0713', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84773e+07, max = 5.53144e+11', '# - flop rate = 11.9526 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49551e-11\trel.res = 1.79134e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.124838', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 2.29409 GFlop/s', '# - bytes moved = 432.807 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.46694 GByte/s', '# - solve arithmetic intensity = 0.661703 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.110s; elapsed, 61.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.880s; elapsed, 61.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.700s; elapsed, 61.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 111.080s; elapsed, 61.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 70.060s; elapsed, 61.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 85.820s; elapsed, 61.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 83.090s; elapsed, 61.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 116.870s; elapsed, 61.324s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 70.060s; elapsed, 61.308s, #calls: 1', 'TOTAL : CPU, 70.060s; elapsed, 61.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.110s; elapsed, 61.316s, #calls: 1', 'TOTAL : CPU, 68.110s; elapsed, 61.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 83.090s; elapsed, 61.308s, #calls: 1', 'TOTAL : CPU, 83.090s; elapsed, 61.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 85.820s; elapsed, 61.308s, #calls: 1', 'TOTAL : CPU, 85.820s; elapsed, 61.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.880s; elapsed, 61.314s, #calls: 1', 'TOTAL : CPU, 68.880s; elapsed, 61.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.700s; elapsed, 61.316s, #calls: 1', 'TOTAL : CPU, 68.700s; elapsed, 61.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 111.080s; elapsed, 61.317s, #calls: 1', 'TOTAL : CPU, 111.080s; elapsed, 61.317s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 237.010s; elapsed, 119.786s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.283s, #calls: 1', 'time for EIGEN_LDLT: CPU, 334.340s; elapsed, 334.452s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 116.870s; elapsed, 61.324s, #calls: 1', 'TOTAL : CPU, 688.630s; elapsed, 515.845s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 310.150s; elapsed, 78.638s', 'individual call time for EIGEN_LDLT: CPU, 313.270s; elapsed, 313.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.253s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.613572', '# - matching time = 0.150232', '# - symmetrization time = 0.0639539', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.203158', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 60.7165', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84791e+07, max = 5.53144e+11', '# - flop rate = 11.8256 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49343e-11\trel.res = 1.79085e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.135319', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 2.1164 GFlop/s', '# - bytes moved = 432.824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.19854 GByte/s', '# - solve arithmetic intensity = 0.661677 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 81.900s; elapsed, 61.908s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 84.150s; elapsed, 61.908s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 87.470s; elapsed, 61.903s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 83.740s; elapsed, 61.907s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 225.110s; elapsed, 61.909s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 133.980s; elapsed, 61.903s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 133.120s; elapsed, 61.905s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 233.260s; elapsed, 61.916s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 81.900s; elapsed, 61.908s, #calls: 1', 'TOTAL : CPU, 81.900s; elapsed, 61.908s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 133.120s; elapsed, 61.905s, #calls: 1', 'TOTAL : CPU, 133.120s; elapsed, 61.905s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 83.740s; elapsed, 61.907s, #calls: 1', 'TOTAL : CPU, 83.740s; elapsed, 61.907s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 225.110s; elapsed, 61.909s, #calls: 1', 'TOTAL : CPU, 225.110s; elapsed, 61.909s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 87.470s; elapsed, 61.903s, #calls: 1', 'TOTAL : CPU, 87.470s; elapsed, 61.903s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 84.150s; elapsed, 61.908s, #calls: 1', 'TOTAL : CPU, 84.150s; elapsed, 61.908s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 133.980s; elapsed, 61.903s, #calls: 1', 'TOTAL : CPU, 133.980s; elapsed, 61.903s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 310.150s; elapsed, 78.638s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.253s, #calls: 1', 'time for EIGEN_LDLT: CPU, 313.270s; elapsed, 313.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 233.260s; elapsed, 61.916s, #calls: 1', 'TOTAL : CPU, 857.210s; elapsed, 453.864s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 205.250s; elapsed, 205.762s', 'individual call time for EIGEN_LDLT: CPU, 328.620s; elapsed, 328.831s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.305s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,149', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.640875', '# - matching time = 0.141697', '# - symmetrization time = 0.0501812', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,149', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.173143', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 902.044 MB', '# - factor time = 85.2134', '# - factor nonzeros = 112,755,472', '# - factor memory = 902.044 MB', '# - total flops = 7.1691e+11, min = 9.76849e+07, max = 5.51315e+11', '# - flop rate = 8.41311 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2397e+11', '# --------------------------------------------', '# total = 7.2397e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.99952e-11\trel.res = 1.91179e-15\tbw.error = 5.68886e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.10624', '# - total flops = 3.67095e+08, min = 217887, max = 1.87487e+08', '# - flop rate = 3.45533 GFlop/s', '# - bytes moved = 405.893 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.82053 GByte/s', '# - solve arithmetic intensity = 0.904413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 86.250s; elapsed, 86.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 86.270s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 86.270s; elapsed, 86.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 86.240s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 86.310s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 86.230s; elapsed, 86.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 86.210s; elapsed, 86.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 86.290s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.280s; elapsed, 86.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 86.300s; elapsed, 86.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 86.300s; elapsed, 86.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 86.240s; elapsed, 86.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 86.240s; elapsed, 86.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 86.230s; elapsed, 86.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 86.280s; elapsed, 86.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.100s; elapsed, 86.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 86.230s; elapsed, 86.333s, #calls: 1', 'TOTAL : CPU, 86.230s; elapsed, 86.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 86.290s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.290s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 86.270s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.270s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 86.310s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.310s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 86.240s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.240s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 86.250s; elapsed, 86.334s, #calls: 1', 'TOTAL : CPU, 86.250s; elapsed, 86.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 86.270s; elapsed, 86.336s, #calls: 1', 'TOTAL : CPU, 86.270s; elapsed, 86.336sExiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 86.300s; elapsed, 86.333s, #calls: 1', 'TOTAL : CPU, 86.300s; elapsed, 86.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 86.240s; elapsed, 86.332s, #calls: 1', 'TOTAL : CPU, 86.240s; elapsed, 86.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.280s; elapsed, 86.334s, #calls: 1', 'TOTAL : CPU, 86.280s; elapsed, 86.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 86.300s; elapsed, 86.333s, #calls: 1', 'TOTAL : CPU, 86.300s; elapsed, 86.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 86.240s; elapsed, 86.332s, #calls: 1', 'TOTAL : CPU, 86.240s; elapsed, 86.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 86.230s; elapsed, 86.336s, #calls: 1', 'TOTAL : CPU, 86.230s; elapsed, 86.336s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 86.210s; elapsed, 86.332s, #calls: 1', 'TOTAL : CPU, 86.210s; elapsed, 86.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 86.280s; elapsed, 86.336s, #calls: 1', 'TOTAL : CPU, 86.280s; elapsed, 86.336s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 205.250s; elapsed, 205.762s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.305s, #calls: 1', 'time for EIGEN_LDLT: CPU, 328.620s; elapsed, 328.831s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.100s; elapsed, 86.342s, #calls: 1', 'TOTAL : CPU, 620.280s; elapsed, 621.240s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 187.000s; elapsed, 94.271s', 'individual call time for EIGEN_LDLT: CPU, 303.190s; elapsed, 303.165s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.240s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,149', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.632178', '# - matching time = 0.149364', '# - symmetrization time = 0.053596', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,149', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.184642', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 902.044 MB', '# - factor time = 59.7917', '# - factor nonzeros = 112,755,472', '# - factor memory = 902.044 MB', '# - total flops = 7.1691e+11, min = 9.76849e+07, max = 5.51315e+11', '# - flop rate = 11.9901 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2397e+11', '# --------------------------------------------', '# total = 7.2397e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.98842e-11\trel.res = 1.90914e-15\tbw.error = 5.67767e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.097754', '# - total flops = 3.67095e+08, min = 217887, max = 1.87487e+08', '# - flop rate = 3.75529 GFlop/s', '# - bytes moved = 405.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.15228 GByte/s', '# - solve arithmetic intensity = 0.904391 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.350s; elapsed, 60.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 67.360s; elapsed, 60.925s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 67.150s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 67.490s; elapsed, 60.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 66.760s; elapsed, 60.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 65.530s; elapsed, 60.927s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 68.260s; elapsed, 60.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 66.220s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 66.400s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 65.540s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 68.350s; elapsed, 60.927s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 106.640s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 82.150s; elapsed, 60.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 73.550s; elapsed, 60.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 78.890s; elapsed, 60.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.100s; elapsed, 60.932s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.350s; elapsed, 60.928s, #calls: 1', 'TOTAL : CPU, 67.350s; elapsed, 60.928s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 67.150s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 67.150s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 67.490s; elapsed, 60.924s, #calls: 1', 'TOTAL : CPU, 67.490s; elapsed, 60.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 66.220s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 66.220s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 65.530s; elapsed, 60.927s, #calls: 1', 'TOTAL : CPU, 65.530s; elapsed, 60.927s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 73.550s; elapsed, 60.924s, #calls: 1', 'TOTAL : CPU, 73.550s; elapsed, 60.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 67.360s; elapsed, 60.925s, #calls: 1', 'TOTAL : CPU, 67.360s; elapsed, 60.925s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 68.260s; elapsed, 60.922s, #calls: 1', 'TOTAL : CPU, 68.260s; elapsed, 60.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 65.540s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 65.540s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 66.760s; elapsed, 60.923s, #calls: 1', 'TOTAL : CPU, 66.760s; elapsed, 60.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 66.400s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 66.400s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 82.150s; elapsed, 60.922s, #calls: 1', 'TOTAL : CPU, 82.150s; elapsed, 60.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 106.640s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 106.640s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 78.890s; elapsed, 60.923s, #calls: 1', 'TOTAL : CPU, 78.890s; elapsed, 60.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 68.350s; elapsed, 60.927s, #calls: 1', 'TOTAL : CPU, 68.350s; elapsed, 60.927s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 187.000s; elapsed, 94.271s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.240s, #calls: 1', 'time for EIGEN_LDLT: CPU, 303.190s; elapsed, 303.165s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.100s; elapsed, 60.932s, #calls: 1', 'TOTAL : CPU, 605.650s; elapsed, 458.608s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_10k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_10k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 42.720s; elapsed, 42.753s', 'individual call time for EIGEN_LDLT: CPU, 37.930s; elapsed, 37.930s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 42.720s; elapsed, 42.753s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.930s; elapsed, 37.930s, #calls: 1', 'TOTAL : CPU, 80.760s; elapsed, 80.798s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 50.110s; elapsed, 25.125s', 'individual call time for EIGEN_LDLT: CPU, 37.790s; elapsed, 37.624s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 50.110s; elapsed, 25.125s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.790s; elapsed, 37.624s, #calls: 1', 'TOTAL : CPU, 88.060s; elapsed, 62.860s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 70.530s; elapsed, 17.760s', 'individual call time for EIGEN_LDLT: CPU, 38.120s; elapsed, 37.624s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.104s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 70.530s; elapsed, 17.760s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.120s; elapsed, 37.624s, #calls: 1', 'TOTAL : CPU, 108.860s; elapsed, 55.488s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 141.210s; elapsed, 17.809s', 'individual call time for EIGEN_LDLT: CPU, 38.910s; elapsed, 37.721s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.090s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 141.210s; elapsed, 17.809s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.090s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.910s; elapsed, 37.721s, #calls: 1', 'TOTAL : CPU, 180.380s; elapsed, 55.620s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 296.500s; elapsed, 18.642s', 'individual call time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.716s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.088s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 296.500s; elapsed, 18.642s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.716s, #calls: 1', 'TOTAL : CPU, 337.250s; elapsed, 56.446s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 44.830s; elapsed, 44.868s', 'individual call time for EIGEN_LDLT: CPU, 39.250s; elapsed, 39.298s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.260154', '# - matching time = 0.0706661', '# - symmetrization time = 0.047965', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.08951', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 29.5005', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 3.30325 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.93647e-11\trel.res = 2.39962e-15\tbw.error = 4.56252e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.111981', '# - total flops = 6.69055e+07, min = 2.31678e+07, max = 4.37377e+07', '# - flop rate = 0.597472 GFlop/s', '# - bytes moved = 286.981 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56277 GByte/s', '# - solve arithmetic intensity = 0.233135 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.080s; elapsed, 30.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.110s; elapsed, 30.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.080s; elapsed, 30.117s, #calls: 1', 'TOTAL : CPU, 30.080s; elapsed, 30.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 44.830s; elapsed, 44.868s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.250s; elapsed, 39.298s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.110s; elapsed, 30.133s, #calls: 1', 'TOTAL : CPU, 114.310s; elapsed, 114.421s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 52.280s; elapsed, 26.230s', 'individual call time for EIGEN_LDLT: CPU, 38.910s; elapsed, 38.767s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.245893', '# - matching time = 0.0730791', '# - symmetrization time = 0.0460291', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0892651', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 17.9068', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 5.44195 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.91496e-11\trel.res = 2.39443e-15\tbw.error = 4.6452e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120273', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.556339 GFlop/s', '# - bytes moved = 287.3 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.38874 GByte/s', '# - solve arithmetic intensity = 0.232901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.740s; elapsed, 18.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.750s; elapsed, 18.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.740s; elapsed, 18.523s, #calls: 1', 'TOTAL : CPU, 32.740s; elapsed, 18.523s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 52.280s; elapsed, 26.230s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.910s; elapsed, 38.767s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.750s; elapsed, 18.536s, #calls: 1', 'TOTAL : CPU, 128.100s; elapsed, 83.645s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 71.530s; elapsed, 18.036s', 'individual call time for EIGEN_LDLT: CPU, 38.140s; elapsed, 37.618s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.101s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.240442', '# - matching time = 0.0652699', '# - symmetrization time = 0.0480769', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0959878', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 18.0631', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 5.39485 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.90918e-11\trel.res = 2.39303e-15\tbw.error = 4.6452e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.127596', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.524409 GFlop/s', '# - bytes moved = 287.316 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25177 GByte/s', '# - solve arithmetic intensity = 0.232888 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.010s; elapsed, 18.685s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.740s; elapsed, 18.700s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.010s; elapsed, 18.685s, #calls: 1', 'TOTAL : CPU, 66.010s; elapsed, 18.685s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 71.530s; elapsed, 18.036s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.140s; elapsed, 37.618s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.740s; elapsed, 18.700s, #calls: 1', 'TOTAL : CPU, 183.590s; elapsed, 74.455s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 143.700s; elapsed, 18.136s', 'individual call time for EIGEN_LDLT: CPU, 40.210s; elapsed, 39.005s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.269401', '# - matching time = 0.0709729', '# - symmetrization time = 0.049592', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.102719', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 18.6305', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 5.23055 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9097e-11\trel.res = 2.39316e-15\tbw.error = 4.64314e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.118326', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.565493 GFlop/s', '# - bytes moved = 287.348 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.42845 GByte/s', '# - solve arithmetic intensity = 0.232862 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.660s; elapsed, 19.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.700s; elapsed, 19.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.660s; elapsed, 19.284s, #calls: 1', 'TOTAL : CPU, 120.660s; elapsed, 19.284s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 143.700s; elapsed, 18.136s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.210s; elapsed, 39.005s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.700s; elapsed, 19.299s, #calls: 1', 'TOTAL : CPU, 335.850s; elapsed, 76.544s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 324.760s; elapsed, 20.429s', 'individual call time for EIGEN_LDLT: CPU, 48.880s; elapsed, 46.209s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.272035', '# - matching time = 0.0876808', '# - symmetrization time = 0.0576181', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.11199', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 19.994', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 4.87385 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.90639e-11\trel.res = 2.39236e-15\tbw.error = 4.64314e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.137957', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.485024 GFlop/s', '# - bytes moved = 287.413 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08335 GByte/s', '# - solve arithmetic intensity = 0.232809 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.560s; elapsed, 20.713s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 325.340s; elapsed, 20.726s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.560s; elapsed, 20.713s, #calls: 1', 'TOTAL : CPU, 243.560s; elapsed, 20.713s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 324.760s; elapsed, 20.429s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 48.880s; elapsed, 46.209s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 325.340s; elapsed, 20.726s, #calls: 1', 'TOTAL : CPU, 699.380s; elapsed, 87.463s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 52.530s; elapsed, 52.625s', 'individual call time for EIGEN_LDLT: CPU, 38.690s; elapsed, 38.712s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.258646', '# - matching time = 0.0747569', '# - symmetrization time = 0.037302', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.077363', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 18.0914', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73096e+06, max = 5.90189e+10', '# - flop rate = 5.32337 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49226e-11\trel.res = 1.80935e-15\tbw.error = 4.2417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0825031', '# - total flops = 7.3905e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.895785 GFlop/s', '# - bytes moved = 252.69 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.06279 GByte/s', '# - solve arithmetic intensity = 0.292473 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.610s; elapsed, 18.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.610s; elapsed, 18.633s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.630s; elapsed, 18.645s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.630s; elapsed, 18.652s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.610s; elapsed, 18.647s, #calls: 1', 'TOTAL : CPU, 18.610s; elapsed, 18.647s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.610s; elapsed, 18.633s, #calls: 1', 'TOTAL : CPU, 18.610s; elapsed, 18.633s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.630s; elapsed, 18.645s, #calls: 1', 'TOTAL : CPU, 18.630s; elapsed, 18.645s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 52.530s; elapsed, 52.625s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.690s; elapsed, 38.712s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.630s; elapsed, 18.652s, #calls: 1', 'TOTAL : CPU, 109.980s; elapsed, 110.116s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.490s; elapsed, 27.841s', 'individual call time for EIGEN_LDLT: CPU, 39.860s; elapsed, 39.709s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4CPP -2', 'CPP -2', 'CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# - nd time = # As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.300268', '# - matching time = 0.079905', '# - symmetrization time = 0.0366862', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.091177', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 17.6616', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73096e+06, max = 5.90189e+10', '# - flop rate = 5.45292 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47973e-11\trel.res = 1.80633e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0819921', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.901478 GFlop/s', '# - bytes moved = 252.882 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.08422 GByte/s', '# - solve arithmetic intensity = 0.292287 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 27.570s; elapsed, 18.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.520s; elapsed, 18.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 32.340s; elapsed, 18.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 35.960s; elapsed, 18.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 27.570s; elapsed, 18.282s, #calls: 1', 'TOTAL : CPU, 27.570s; elapsed, 18.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.520s; elapsed, 18.278s, #calls: 1', 'TOTAL : CPU, 20.520s; elapsed, 18.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 32.340s; elapsed, 18.270s, #calls: 1', 'TOTAL : CPU, 32.340s; elapsed, 18.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.490s; elapsed, 27.841s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.860s; elapsed, 39.709s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 35.960s; elapsed, 18.286s, #calls: 1', 'TOTAL : CPU, 131.460s; elapsed, 85.947s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 71.350s; elapsed, 17.954s', 'individual call time for EIGEN_LDLT: CPU, 38.790s; elapsed, 38.277s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5CPP -2', 'CPP -2', 'CPP -2', ' = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.272886', '# - matching time = 0.078949', '# - symmetrization time = 0.0410912', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0950689', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 17.7718', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73175e+06, max = 5.90189e+10', '# - flop rate = 5.4191 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47955e-11\trel.res = 1.80628e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0979948', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.754266 GFlop/s', '# - bytes moved = 252.89 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.58065 GByte/s', '# - solve arithmetic intensity = 0.292277 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 47.850s; elapsed, 18.389s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.760s; elapsed, 18.388s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 60.490s; elapsed, 18.377s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.150s; elapsed, 18.394s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.760s; elapsed, 18.388s, #calls: 1', 'TOTAL : CPU, 23.760s; elapsed, 18.388s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 47.850s; elapsed, 18.389s, #calls: 1', 'TOTAL : CPU, 47.850s; elapsed, 18.389s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 60.490s; elapsed, 18.377s, #calls: 1', 'TOTAL : CPU, 60.490s; elapsed, 18.377s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 71.350s; elapsed, 17.954s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.790s; elapsed, 38.277s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.150s; elapsed, 18.394s, #calls: 1', 'TOTAL : CPU, 182.470s; elapsed, 74.721s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.310s; elapsed, 18.945s', 'individual call time for EIGEN_LDLT: CPU, 39.170s; elapsed, 37.967s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.283675', '# - matching time = 0.0831909', '# - symmetrization time = 0.0405788', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.102354', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 18.8719', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73175e+06, max = 5.90189e+10', '# - flop rate = 5.1032 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47952e-11\trel.res = 1.80628e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.101359', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.72923 GFlop/s', '# - bytes moved = 252.909 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.49517 GByte/s', '# - solve arithmetic intensity = 0.292256 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 89.210s; elapsed, 19.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.930s; elapsed, 19.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 120.870s; elapsed, 19.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.950s; elapsed, 19.519s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.930s; elapsed, 19.511s, #calls: 1', 'TOTAL : CPU, 31.930s; elapsed, 19.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 120.870s; elapsed, 19.502s, #calls: 1', 'TOTAL : CPU, 120.870s; elapsed, 19.502s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 89.210s; elapsed, 19.513s, #calls: 1', 'TOTAL : CPU, 89.210s; elapsed, 19.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.310s; elapsed, 18.945s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.170s; elapsed, 37.967s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.950s; elapsed, 19.519s, #calls: 1', 'TOTAL : CPU, 341.700s; elapsed, 76.550s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 45.960s; elapsed, 45.975s', 'individual call time for EIGEN_LDLT: CPU, 39.940s; elapsed, 39.948s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.266578', '# - matching time = 0.0708241', '# - symmetrization time = 0.037334', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0776591', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 19.8297', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33333e+06, max = 5.89486e+10', '# - flop rate = 4.92318 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.77384e-11\trel.res = 1.63586e-15\tbw.error = 4.40647e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079818', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 1.15118 GFlop/s', '# - bytes moved = 243.937 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.05617 GByte/s', '# - solve arithmetic intensity = 0.376674 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.340s; elapsed, 20.385s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.340s; elapsed, 20.381s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.370s; elapsed, 20.382s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.360s; elapsed, 20.382s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.340s; elapsed, 20.380s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.360s; elapsed, 20.380s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.350s; elapsed, 20.370s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.370s; elapsed, 20.388s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.360s; elapsed, 20.382s, #calls: 1', 'TOTAL : CPU, 20.360s; elapsed, 20.382s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.340s; elapsed, 20.381s, #calls: 1', 'TOTAL : CPU, 20.340s; elapsed, 20.381s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.370s; elapsed, 20.382s, #calls: 1', 'TOTAL : CPU, 20.370s; elapsed, 20.382s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.340s; elapsed, 20.380s, #calls: 1', 'TOTAL : CPU, 20.340s; elapsed, 20.380s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.340s; elapsed, 20.385s, #calls: 1', 'TOTAL : CPU, 20.340s; elapsed, 20.385s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.360s; elapsed, 20.380s, #calls: 1', 'TOTAL : CPU, 20.360s; elapsed, 20.380s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.350s; elapsed, 20.370s, #calls: 1', 'TOTAL : CPU, 20.350s; elapsed, 20.370s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 45.960s; elapsed, 45.975s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.940s; elapsed, 39.948s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.370s; elapsed, 20.388s, #calls: 1', 'TOTAL : CPU, 106.400s; elapsed, 106.439s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 57.900s; elapsed, 29.060s', 'individual call time for EIGEN_LDLT: CPU, 41.590s; elapsed, 41.450s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', ' = log_2(#threads) + 3', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.302425', '# - matching time = 0.0819871', '# - symmetrization time = 0.0387001', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0952189', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 19.7828', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33333e+06, max = 5.89486e+10', '# - flop rate = 4.93485 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.76835e-11\trel.res = 1.63453e-15\tbw.error = 4.41108e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0891449', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 1.03073 GFlop/s', '# - bytes moved = 243.947 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.73652 GByte/s', '# - solve arithmetic intensity = 0.376659 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.120s; elapsed, 20.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.260s; elapsed, 20.416s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 34.220s; elapsed, 20.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 22.930s; elapsed, 20.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 28.780s; elapsed, 20.416s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 22.690s; elapsed, 20.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.490s; elapsed, 20.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.200s; elapsed, 20.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 34.220s; elapsed, 20.415s, #calls: 1', 'TOTAL : CPU, 34.220s; elapsed, 20.415s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.260s; elapsed, 20.416s, #calls: 1', 'TOTAL : CPU, 22.260s; elapsed, 20.416s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.120s; elapsed, 20.422s, #calls: 1', 'TOTAL : CPU, 27.120s; elapsed, 20.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 22.930s; elapsed, 20.405s, #calls: 1', 'TOTAL : CPU, 22.930s; elapsed, 20.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 22.690s; elapsed, 20.418s, #calls: 1', 'TOTAL : CPU, 22.690s; elapsed, 20.418s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 28.780s; elapsed, 20.416s, #calls: 1', 'TOTAL : CPU, 28.780s; elapsed, 20.416s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.490s; elapsed, 20.415s, #calls: 1', 'TOTAL : CPU, 22.490s; elapsed, 20.415s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 57.900s; elapsed, 29.060s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 41.590s; elapsed, 41.450s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.200s; elapsed, 20.422s, #calls: 1', 'TOTAL : CPU, 139.840s; elapsed, 91.039s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 78.000s; elapsed, 19.651s', 'individual call time for EIGEN_LDLT: CPU, 46.670s; elapsed, 46.192s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.276174', '# - matching time = 0.081959', '# - symmetrization time = 0.0441151', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.101362', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 18.8785', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33405e+06, max = 5.89486e+10', '# - flop rate = 5.17125 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.76723e-11\trel.res = 1.63426e-15\tbw.error = 4.40185e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.108999', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 0.842987 GFlop/s', '# - bytes moved = 243.959 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.23818 GByte/s', '# - solve arithmetic intensity = 0.37664 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 37.060s; elapsed, 19.527s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 45.220s; elapsed, 19.520s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.040s; elapsed, 19.522s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 27.170s; elapsed, 19.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 26.640s; elapsed, 19.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.140s; elapsed, 19.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.850s; elapsed, 19.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.390s; elapsed, 19.532s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 27.170s; elapsed, 19.511s, #calls: 1', 'TOTAL : CPU, 27.170s; elapsed, 19.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.040s; elapsed, 19.522s, #calls: 1', 'TOTAL : CPU, 68.040s; elapsed, 19.522s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.850s; elapsed, 19.523s, #calls: 1', 'TOTAL : CPU, 25.850s; elapsed, 19.523s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 45.220s; elapsed, 19.520s, #calls: 1', 'TOTAL : CPU, 45.220s; elapsed, 19.520s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 26.640s; elapsed, 19.525s, #calls: 1', 'TOTAL : CPU, 26.640s; elapsed, 19.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.140s; elapsed, 19.523s, #calls: 1', 'TOTAL : CPU, 25.140s; elapsed, 19.523s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 37.060s; elapsed, 19.527s, #calls: 1', 'TOTAL : CPU, 37.060s; elapsed, 19.527s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 78.000s; elapsed, 19.651s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 46.670s; elapsed, 46.192s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.390s; elapsed, 19.532s, #calls: 1', 'TOTAL : CPU, 201.250s; elapsed, 85.470s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 45.730s; elapsed, 45.773s', 'individual call time for EIGEN_LDLT: CPU, 39.370s; elapsed, 39.397s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.2644', '# - matching time = 0.0711598', '# - symmetrization time = 0.017853', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.069427', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 17.5398', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.58592e+06, max = 5.89302e+10', '# - flop rate = 5.55579 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01444e+11', '# --------------------------------------------', '# total = 1.01444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.46798e-11\trel.res = 2.28648e-15\tbw.error = 3.3904e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0566521', '# - total flops = 1.34224e+08, min = 40130, max = 4.34304e+07', '# - flop rate = 2.36926 GFlop/s', '# - bytes moved = 235.766 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.16166 GByte/s', '# - solve arithmetic intensity = 0.569308 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.000s; elapsed, 18.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.980s; elapsed, 18.038s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.980s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 18.000s; elapsed, 18.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.970s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.990s; elapsed, 18.038s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.030s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.010s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 18.010s; elapsed, 18.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.990s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.960s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.950s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 18.020s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.980s; elapsed, 18.030s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.960s; elapsed, 18.031s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.030s; elapsed, 18.042s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 18.000s; elapsed, 18.034s, #calls: 1', 'TOTAL : CPU, 18.000s; elapsed, 18.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.030s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.000s; elapsed, 18.035s, #calls: 1', 'TOTAL : CPU, 18.000s; elapsed, 18.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.980s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 17.980s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.960s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 17.960s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.980s; elapsed, 18.038s, #calls: 1', 'TOTAL : CPU, 17.980s; elapsed, 18.038s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.990s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 17.990s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.970s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 17.970s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.950s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 17.950s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.010s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 18.010s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.990s; elapsed, 18.038s, #calls: 1', 'TOTAL : CPU, 17.990s; elapsed, 18.038s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 18.020s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 18.020s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.960s; elapsed, 18.031s, #calls: 1', 'TOTAL : CPU, 17.960s; elapsed, 18.031s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 18.010s; elapsed, 18.035s, #calls: 1', 'TOTAL : CPU, 18.010s; elapsed, 18.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.980s; elapsed, 18.030s, #calls: 1', 'TOTAL : CPU, 17.980s; elapsed, 18.030s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 45.730s; elapsed, 45.773s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.370s; elapsed, 39.397s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.030s; elapsed, 18.042s, #calls: 1', 'TOTAL : CPU, 103.260s; elapsed, 103.337s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 56.270s; elapsed, 28.244s', 'individual call time for EIGEN_LDLT: CPU, 40.490s; elapsed, 40.346s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.122s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.306187', '# - matching time = 0.0821331', '# - symmetrization time = 0.0194452', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.087703', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 18.2915', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.58592e+06, max = 5.89302e+10', '# - flop rate = 5.32747 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01444e+11', '# --------------------------------------------', '# total = 1.01444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.46625e-11\trel.res = 2.28606e-15\tbw.error = 3.3904e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0704439', '# - total flops = 1.34224e+08, min = 40130, max = 4.34304e+07', '# - flop rate = 1.9054 GFlop/s', '# - bytes moved = 235.775 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.34699 GByte/s', '# - solve arithmetic intensity = 0.569288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.260s; elapsed, 18.881s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.310s; elapsed, 18.876s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 25.290s; elapsed, 18.880s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 24.320s; elapsed, 18.880s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.180s; elapsed, 18.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 20.920s; elapsed, 18.873s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 29.130s; elapsed, 18.875s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.080s; elapsed, 18.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.850s; elapsed, 18.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.640s; elapsed, 18.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.320s; elapsed, 18.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 33.520s; elapsed, 18.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 21.390s; elapsed, 18.880s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.170s; elapsed, 18.878s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 20.710s; elapsed, 18.874s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.210s; elapsed, 18.884s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.310s; elapsed, 18.876s, #calls: 1', 'TOTAL : CPU, 20.310s; elapsed, 18.876s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 25.290s; elapsed, 18.880s, #calls: 1', 'TOTAL : CPU, 25.290s; elapsed, 18.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 20.920s; elapsed, 18.873s, #calls: 1', 'TOTAL : CPU, 20.920s; elapsed, 18.873s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.180s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 20.180s; elapsed, 18.877s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 24.320s; elapsed, 18.880s, #calls: 1', 'TOTAL : CPU, 24.320s; elapsed, 18.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.260s; elapsed, 18.881s, #calls: 1', 'TOTAL : CPU, 21.260s; elapsed, 18.881s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 21.390s; elapsed, 18.880s, #calls: 1', 'TOTAL : CPU, 21.390s; elapsed, 18.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.850s; elapsed, 18.879s, #calls: 1', 'TOTAL : CPU, 20.850s; elapsed, 18.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 29.130s; elapsed, 18.875s, #calls: 1', 'TOTAL : CPU, 29.130s; elapsed, 18.875s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 20.710s; elapsed, 18.874s, #calls: 1', 'TOTAL : CPU, 20.710s; elapsed, 18.874s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.320s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 20.320s; elapsed, 18.877s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.170s; elapsed, 18.878s, #calls: 1', 'TOTAL : CPU, 21.170s; elapsed, 18.878s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.080s; elapsed, 18.879s, #calls: 1', 'TOTAL : CPU, 21.080s; elapsed, 18.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.640s; elapsed, 18.879s, #calls: 1', 'TOTAL : CPU, 20.640s; elapsed, 18.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 33.520s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 33.520s; elapsed, 18.877s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 56.270s; elapsed, 28.244s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.122s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.490s; elapsed, 40.346s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.210s; elapsed, 18.884s, #calls: 1', 'TOTAL : CPU, 134.140s; elapsed, 87.596s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.810s; elapsed, 30.840s', 'individual call time for EIGEN_LDLT: CPU, 38.180s; elapsed, 38.208s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.810s; elapsed, 30.840s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.180s; elapsed, 38.208s, #calls: 1', 'TOTAL : CPU, 69.110s; elapsed, 69.165s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 41.000s; elapsed, 20.617s', 'individual call time for EIGEN_LDLT: CPU, 38.080s; elapsed, 37.911s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 41.000s; elapsed, 20.617s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.080s; elapsed, 37.911s, #calls: 1', 'TOTAL : CPU, 79.250s; elapsed, 58.644s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 56.380s; elapsed, 14.216s', 'individual call time for EIGEN_LDLT: CPU, 38.250s; elapsed, 37.729s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 56.380s; elapsed, 14.216s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.250s; elapsed, 37.729s, #calls: 1', 'TOTAL : CPU, 94.820s; elapsed, 52.043s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 108.810s; elapsed, 13.681s', 'individual call time for EIGEN_LDLT: CPU, 38.950s; elapsed, 37.672s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.092s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 108.810s; elapsed, 13.681s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.092s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.950s; elapsed, 37.672s, #calls: 1', 'TOTAL : CPU, 147.980s; elapsed, 51.445s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 256.440s; elapsed, 16.123s', 'individual call time for EIGEN_LDLT: CPU, 40.460s; elapsed, 37.713s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 256.440s; elapsed, 16.123s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.460s; elapsed, 37.713s, #calls: 1', 'TOTAL : CPU, 297.300s; elapsed, 53.936s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.690s; elapsed, 30.711s', 'individual call time for EIGEN_LDLT: CPU, 38.040s; elapsed, 38.080s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.185051', '# - matching time = 0.0646098', '# - symmetrization time = 0.039875', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0939381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 21.516', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.69076e+10, max = 7.00936e+10', '# - flop rate = 4.50832 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.66658e-11\trel.res = 1.85145e-15\tbw.error = 5.42862e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0925951', '# - total flops = 6.83649e+07, min = 1.74802e+07, max = 5.08847e+07', '# - flop rate = 0.738321 GFlop/s', '# - bytes moved = 238.86 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57962 GByte/s', '# - solve arithmetic intensity = 0.286213 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.980s; elapsed, 22.006s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.010s; elapsed, 22.026s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.980s; elapsed, 22.006s, #calls: 1', 'TOTAL : CPU, 21.980s; elapsed, 22.006s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.690s; elapsed, 30.711s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.040s; elapsed, 38.080s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.010s; elapsed, 22.026s, #calls: 1', 'TOTAL : CPU, 90.870s; elapsed, 90.934s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.676s', 'individual call time for EIGEN_LDLT: CPU, 39.410s; elapsed, 39.262s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.197675', '# - matching time = 0.0692399', '# - symmetrization time = 0.043412', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.104828', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.6384', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00936e+10', '# - flop rate = 6.20276 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.65231e-11\trel.res = 1.848e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.121836', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.561174 GFlop/s', '# - bytes moved = 239.112 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.96258 GByte/s', '# - solve arithmetic intensity = 0.285937 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 27.620s; elapsed, 16.194s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.950s; elapsed, 16.215s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 27.620s; elapsed, 16.194s, #calls: 1', 'TOTAL : CPU, 27.620s; elapsed, 16.194s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.676s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.410s; elapsed, 39.262s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.950s; elapsed, 16.215s, #calls: 1', 'TOTAL : CPU, 106.640s; elapsed, 73.265s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.970s; elapsed, 14.141s', 'individual call time for EIGEN_LDLT: CPU, 38.290s; elapsed, 37.779s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.186009', '# - matching time = 0.065407', '# - symmetrization time = 0.0403771', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.104122', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.6295', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00936e+10', '# - flop rate = 6.63053 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.65479e-11\trel.res = 1.8486e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0887661', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.77024 GFlop/s', '# - bytes moved = 239.127 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.6939 GByte/s', '# - solve arithmetic intensity = 0.28592 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.930s; elapsed, 15.131s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 59.530s; elapsed, 15.149s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.930s; elapsed, 15.131s, #calls: 1', 'TOTAL : CPU, 41.930s; elapsed, 15.131s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.970s; elapsed, 14.141s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.290s; elapsed, 37.779s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 59.530s; elapsed, 15.149s, #calls: 1', 'TOTAL : CPU, 153.980s; elapsed, 67.168s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 112.670s; elapsed, 14.158s', 'individual call time for EIGEN_LDLT: CPU, 47.010s; elapsed, 45.733s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.205432', '# - matching time = 0.0715861', '# - symmetrization time = 0.041888', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.110281', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.9251', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00937e+10', '# - flop rate = 6.09108 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.65254e-11\trel.res = 1.84806e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.123', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.555863 GFlop/s', '# - bytes moved = 239.156 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94436 GByte/s', '# - solve arithmetic intensity = 0.285885 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 88.670s; elapsed, 16.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 129.540s; elapsed, 16.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 88.670s; elapsed, 16.494s, #calls: 1', 'TOTAL : CPU, 88.670s; elapsed, 16.494s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 112.670s; elapsed, 14.158s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 47.010s; elapsed, 45.733s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 129.540s; elapsed, 16.512s, #calls: 1', 'TOTAL : CPU, 289.450s; elapsed, 76.500s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 233.540s; elapsed, 14.654s', 'individual call time for EIGEN_LDLT: CPU, 40.550s; elapsed, 37.835s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.103s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.21567', '# - matching time = 0.0882251', '# - symmetrization time = 0.04984', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.133991', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 16.6166', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00937e+10', '# - flop rate = 5.83762 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.64901e-11\trel.res = 1.84721e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.105698', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.646854 GFlop/s', '# - bytes moved = 239.215 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26319 GByte/s', '# - solve arithmetic intensity = 0.285815 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 173.960s; elapsed, 17.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 270.210s; elapsed, 17.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 173.960s; elapsed, 17.234s, #calls: 1', 'TOTAL : CPU, 173.960s; elapsed, 17.234s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 233.540s; elapsed, 14.654s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.103s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.550s; elapsed, 37.835s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 270.210s; elapsed, 17.253s, #calls: 1', 'TOTAL : CPU, 544.700s; elapsed, 69.845s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.270s; elapsed, 32.295s', 'individual call time for EIGEN_LDLT: CPU, 39.660s; elapsed, 39.698s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.121s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.203172', '# - matching time = 0.0693901', '# - symmetrization time = 0.0527458', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0828469', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 16.2377', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00681e+10', '# - flop rate = 5.97383 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.94624e-11\trel.res = 2.40198e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0929391', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.780964 GFlop/s', '# - bytes moved = 220.307 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.37045 GByte/s', '# - solve arithmetic intensity = 0.329459 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.720s; elapsed, 16.746s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.750s; elapsed, 16.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.750s; elapsed, 16.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.750s; elapsed, 16.766s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.720s; elapsed, 16.746s, #calls: 1', 'TOTAL : CPU, 16.720s; elapsed, 16.746s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.750s; elapsed, 16.761s, #calls: 1', 'TOTAL : CPU, 16.750s; elapsed, 16.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.750s; elapsed, 16.761s, #calls: 1', 'TOTAL : CPU, 16.750s; elapsed, 16.761s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.270s; elapsed, 32.295s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.660s; elapsed, 39.698s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.750s; elapsed, 16.766s, #calls: 1', 'TOTAL : CPU, 88.810s; elapsed, 88.880s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.730s; elapsed, 19.924s', 'individual call time for EIGEN_LDLT: CPU, 38.510s; elapsed, 38.339s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.199401', '# - matching time = 0.068573', '# - symmetrization time = 0.059953', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0962582', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.2064', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00681e+10', '# - flop rate = 6.82797 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.94247e-11\trel.res = 2.40107e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.115793', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.626827 GFlop/s', '# - bytes moved = 220.321 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.90271 GByte/s', '# - solve arithmetic intensity = 0.329439 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.120s; elapsed, 14.756s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.990s; elapsed, 14.774s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.240s; elapsed, 14.772s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.970s; elapsed, 14.780s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.990s; elapsed, 14.774s, #calls: 1', 'TOTAL : CPU, 24.990s; elapsed, 14.774s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.120s; elapsed, 14.756s, #calls: 1', 'TOTAL : CPU, 22.120s; elapsed, 14.756s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.240s; elapsed, 14.772s, #calls: 1', 'TOTAL : CPU, 17.240s; elapsed, 14.772s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.730s; elapsed, 19.924s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.510s; elapsed, 38.339s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.970s; elapsed, 14.780s, #calls: 1', 'TOTAL : CPU, 107.370s; elapsed, 73.153s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.470s; elapsed, 13.954s', 'individual call time for EIGEN_LDLT: CPU, 39.150s; elapsed, 38.612s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.200997', '# - matching time = 0.0708141', '# - symmetrization time = 0.052597', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096231', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 13.8969', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55441e+07, max = 7.00681e+10', '# - flop rate = 6.98007 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9437e-11\trel.res = 2.40137e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.126255', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.574885 GFlop/s', '# - bytes moved = 220.338 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.74518 GByte/s', '# - solve arithmetic intensity = 0.329413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.160s; elapsed, 14.467s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 32.420s; elapsed, 14.455s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 41.960s; elapsed, 14.469s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 56.070s; elapsed, 14.475s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.160s; elapsed, 14.467s, #calls: 1', 'TOTAL : CPU, 21.160s; elapsed, 14.467s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 32.420s; elapsed, 14.455s, #calls: 1', 'TOTAL : CPU, 32.420s; elapsed, 14.455s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 41.960s; elapsed, 14.469s, #calls: 1', 'TOTAL : CPU, 41.960s; elapsed, 14.469s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.470s; elapsed, 13.954s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.150s; elapsed, 38.612s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 56.070s; elapsed, 14.475s, #calls: 1', 'TOTAL : CPU, 150.870s; elapsed, 67.147s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 144.060s; elapsed, 18.234s', 'individual call time for EIGEN_LDLT: CPU, 40.790s; elapsed, 39.540s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.097s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.236389', '# - matching time = 0.080327', '# - symmetrization time = 0.0596118', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.114193', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.7271', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55442e+07, max = 7.00681e+10', '# - flop rate = 6.58658 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.94182e-11\trel.res = 2.40091e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.117395', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.618273 GFlop/s', '# - bytes moved = 220.371 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87718 GByte/s', '# - solve arithmetic intensity = 0.329363 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.480s; elapsed, 15.347s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.560s; elapsed, 15.365s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 87.830s; elapsed, 15.364s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.690s; elapsed, 15.369s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.480s; elapsed, 15.347s, #calls: 1', 'TOTAL : CPU, 59.480s; elapsed, 15.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.560s; elapsed, 15.365s, #calls: 1', 'TOTAL : CPU, 28.560s; elapsed, 15.365s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 87.830s; elapsed, 15.364s, #calls: 1', 'TOTAL : CPU, 87.830s; elapsed, 15.364s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 144.060s; elapsed, 18.234s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.097s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.790s; elapsed, 39.540s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.690s; elapsed, 15.369s, #calls: 1', 'TOTAL : CPU, 303.790s; elapsed, 73.241s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 2541.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.880s; elapsed, 36.943s', 'individual call time for EIGEN_LDLT: CPU, 38.530s; elapsed, 38.563s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.195898', '# - matching time = 0.0672462', '# - symmetrization time = 0.0305309', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0815339', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 16.6121', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00086e+10', '# - flop rate = 5.83921 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.915e+10', '# --------------------------------------------', '# total = 9.915e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02125e-10\trel.res = 2.46627e-15\tbw.error = 4.30241e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.081059', '# - total flops = 8.29426e+07, min = 120436, max = 5.04792e+07', '# - flop rate = 1.02324 GFlop/s', '# - bytes moved = 210.5 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.59687 GByte/s', '# - solve arithmetic intensity = 0.394027 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.060s; elapsed, 17.087s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.030s; elapsed, 17.088s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.060s; elapsed, 17.079s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.040s; elapsed, 17.088s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 17.087s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.070s; elapsed, 17.090s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.060s; elapsed, 17.082s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.060s; elapsed, 17.093s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 17.087s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.087s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.030s; elapsed, 17.088s, #calls: 1', 'TOTAL : CPU, 17.030s; elapsed, 17.088s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.060s; elapsed, 17.079s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.079s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.040s; elapsed, 17.088s, #calls: 1', 'TOTAL : CPU, 17.040s; elapsed, 17.088s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.060s; elapsed, 17.087s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.087s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.070s; elapsed, 17.090s, #calls: 1', 'TOTAL : CPU, 17.070s; elapsed, 17.090s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.060s; elapsed, 17.082s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.082s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.880s; elapsed, 36.943s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.530s; elapsed, 38.563s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.060s; elapsed, 17.093s, #calls: 1', 'TOTAL : CPU, 92.580s; elapsed, 92.711s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 2541.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.660s; elapsed, 17.872s', 'individual call time for EIGEN_LDLT: CPU, 39.590s; elapsed, 39.418s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.217309', '# - matching time = 0.0769148', '# - symmetrization time = 0.0293219', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096776', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.8886', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00086e+10', '# - flop rate = 6.51514 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.915e+10', '# --------------------------------------------', '# total = 9.915e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02122e-10\trel.res = 2.46621e-15\tbw.error = 4.30241e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.077673', '# - total flops = 8.29426e+07, min = 120436, max = 5.04792e+07', '# - flop rate = 1.06784 GFlop/s', '# - bytes moved = 210.509 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71019 GByte/s', '# - solve arithmetic intensity = 0.394011 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.130s; elapsed, 15.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.580s; elapsed, 15.398s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 25.620s; elapsed, 15.406s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.740s; elapsed, 15.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.180s; elapsed, 15.397s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.600s; elapsed, 15.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.430s; elapsed, 15.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.190s; elapsed, 15.409s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 25.620s; elapsed, 15.406s, #calls: 1', 'TOTAL : CPU, 25.620s; elapsed, 15.406s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.130s; elapsed, 15.404s, #calls: 1', 'TOTAL : CPU, 17.130s; elapsed, 15.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.580s; elapsed, 15.398s, #calls: 1', 'TOTAL : CPU, 20.580s; elapsed, 15.398s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.740s; elapsed, 15.405s, #calls: 1', 'TOTAL : CPU, 17.740s; elapsed, 15.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.180s; elapsed, 15.397s, #calls: 1', 'TOTAL : CPU, 21.180s; elapsed, 15.397s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.600s; elapsed, 15.404s, #calls: 1', 'TOTAL : CPU, 17.600s; elapsed, 15.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.430s; elapsed, 15.404s, #calls: 1', 'TOTAL : CPU, 17.430s; elapsed, 15.404s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.660s; elapsed, 17.872s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.590s; elapsed, 39.418s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.190s; elapsed, 15.409s, #calls: 1', 'TOTAL : CPU, 105.580s; elapsed, 72.808s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 2541.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 65.430s; elapsed, 16.503s', 'individual call time for EIGEN_LDLT: CPU, 39.240s; elapsed, 38.716s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.104s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.233801', '# - matching time = 0.0823982', '# - symmetrization time = 0.0446789', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.099885', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.9705', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55441e+07, max = 7.00086e+10', '# - flop rate = 6.07379 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.915e+10', '# --------------------------------------------', '# total = 9.915e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02113e-10\trel.res = 2.46598e-15\tbw.error = 4.30241e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0909791', '# - total flops = 8.29426e+07, min = 120436, max = 5.04792e+07', '# - flop rate = 0.911666 GFlop/s', '# - bytes moved = 210.519 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.31393 GByte/s', '# - solve arithmetic intensity = 0.39399 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.950s; elapsed, 16.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.500s; elapsed, 16.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.780s; elapsed, 16.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.070s; elapsed, 16.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 46.160s; elapsed, 16.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 34.300s; elapsed, 16.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.740s; elapsed, 16.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.170s; elapsed, 16.550s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 46.160s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 46.160s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 34.300s; elapsed, 16.535s, #calls: 1', 'TOTAL : CPU, 34.300s; elapsed, 16.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.780s; elapsed, 16.544s, #calls: 1', 'TOTAL : CPU, 22.780s; elapsed, 16.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.070s; elapsed, 16.544s, #calls: 1', 'TOTAL : CPU, 22.070s; elapsed, 16.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.500s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 23.500s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.950s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 23.950s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.740s; elapsed, 16.535s, #calls: 1', 'TOTAL : CPU, 30.740s; elapsed, 16.535s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 65.430s; elapsed, 16.503s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.240s; elapsed, 38.716s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.170s; elapsed, 16.550s, #calls: 1', 'TOTAL : CPU, 169.020s; elapsed, 71.873s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 1270.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.910s; elapsed, 33.928s', 'individual call time for EIGEN_LDLT: CPU, 40.000s; elapsed, 40.019s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.225481', '# - matching time = 0.0781739', '# - symmetrization time = 0.0169551', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0908329', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 13.6496', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.37245e+07, max = 6.98999e+10', '# - flop rate = 7.10653 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.8763e+10', '# --------------------------------------------', '# total = 9.8763e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.17627e-11\trel.res = 2.21604e-15\tbw.error = 4.05167e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.052248', '# - total flops = 1.04874e+08, min = 110416, max = 4.99919e+07', '# - flop rate = 2.00723 GFlop/s', '# - bytes moved = 201.842 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.86315 GByte/s', '# - solve arithmetic intensity = 0.519584 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.100s; elapsed, 14.124s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.070s; elapsed, 14.128s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.100s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.050s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.050s; elapsed, 14.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.070s; elapsed, 14.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.060s; elapsed, 14.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.090s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.110s; elapsed, 14.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.070s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.090s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.090s; elapsed, 14.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.100s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.100s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.080s; elapsed, 14.124s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.120s; elapsed, 14.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.100s; elapsed, 14.124s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.090s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.090s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.070s; elapsed, 14.128s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 14.128s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.050s; elapsed, 14.129s, #calls: 1', 'TOTAL : CPU, 14.050s; elapsed, 14.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.110s; elapsed, 14.126s, #calls: 1', 'TOTAL : CPU, 14.110s; elapsed, 14.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.090s; elapsed, 14.127s, #calls: 1Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.060s; elapsed, 14.123s, #calls: 1', 'TOTAL : CPU, 14.060s; elapsed, 14.123s', '', 'TOTAL : CPU, 14.090s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.100s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.070s; elapsed, 14.122s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 14.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.050s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.050s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.100s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.090s; elapsed, 14.126s, #calls: 1', 'TOTAL : CPU, 14.090s; elapsed, 14.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.100s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.080s; elapsed, 14.124s, #calls: 1', 'TOTAL : CPU, 14.080s; elapsed, 14.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.070s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 14.127s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.910s; elapsed, 33.928s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.131s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.000s; elapsed, 40.019s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.120s; elapsed, 14.133s, #calls: 1', 'TOTAL : CPU, 88.150s; elapsed, 88.211s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 1270.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 46.200s; elapsed, 23.264s', 'individual call time for EIGEN_LDLT: CPU, 42.770s; elapsed, 42.625s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.129s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.229821', '# - matching time = 0.0801702', '# - symmetrization time = 0.020319', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0992281', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.4485', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.37245e+07, max = 6.98999e+10', '# - flop rate = 6.279 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.8763e+10', '# --------------------------------------------', '# total = 9.8763e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.17589e-11\trel.res = 2.21594e-15\tbw.error = 4.05167e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.064244', '# - total flops = 1.04874e+08, min = 110416, max = 4.99919e+07', '# - flop rate = 1.63243 GFlop/s', '# - bytes moved = 201.847 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.14188 GByte/s', '# - solve arithmetic intensity = 0.519571 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.420s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.410s; elapsed, 15.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.150s; elapsed, 15.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 20.330s; elapsed, 15.953s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.660s; elapsed, 15.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.360s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.960s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.690s; elapsed, 15.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.760s; elapsed, 15.960s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 25.760s; elapsed, 15.960s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.560s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 18.220s; elapsed, 15.957s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 18.420s; elapsed, 15.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 20.020s; elapsed, 15.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.330s; elapsed, 15.966s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.420s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 17.420s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 25.760s; elapsed, 15.960s, #calls: 1', 'TOTAL : CPU, 25.760s; elapsed, 15.960s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.690s; elapsed, 15.954s, #calls: 1', 'TOTAL : CPU, 21.690s; elapsed, 15.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.410s; elapsed, 15.959s, #calls: 1', 'TOTAL : CPU, 17.410s; elapsed, 15.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.660s; elapsed, 15.959s, #calls: 1', 'TOTAL : CPU, 18.660s; elapsed, 15.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.760s; elapsed, 15.960s, #calls: 1', 'TOTAL : CPU, 17.760s; elapsed, 15.960s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.150s; elapsed, 15.959s, #calls: 1', 'TOTAL : CPU, 18.150s; elapsed, 15.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.960s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 17.960s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.560s; elapsed, 15.958s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 18.420s; elapsed, 15.954s, #calls: 1', 'TOTAL : CPU, 18.420s; elapsed, 15.954s', 'TOTAL : CPU, 18.560s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 20.020s; elapsed, 15.954s, #calls: 1', 'TOTAL : CPU, 20.020s; elapsed, 15.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 20.330s; elapsed, 15.953s, #calls: 1', 'TOTAL : CPU, 20.330s; elapsed, 15.953s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.360s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 18.360s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 17.730s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 18.220s; elapsed, 15.957s, #calls: 1', 'TOTAL : CPU, 18.220s; elapsed, 15.957s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 46.200s; elapsed, 23.264s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.129s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.770s; elapsed, 42.625s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.330s; elapsed, 15.966s, #calls: 1', 'TOTAL : CPU, 120.480s; elapsed, 81.984s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 142.860s; elapsed, 142.986s', 'individual call time for EIGEN_LDLT: CPU, 292.710s; elapsed, 292.887s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.282s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 142.860s; elapsed, 142.986s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.282s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.710s; elapsed, 292.887s, #calls: 1', 'TOTAL : CPU, 435.860s; elapsed, 436.155s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 182.690s; elapsed, 92.966s', 'individual call time for EIGEN_LDLT: CPU, 292.910s; elapsed, 292.937s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.250s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 182.690s; elapsed, 92.966s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.250s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.910s; elapsed, 292.937s, #calls: 1', 'TOTAL : CPU, 475.990s; elapsed, 386.153s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 326.740s; elapsed, 83.028s', 'individual call time for EIGEN_LDLT: CPU, 293.430s; elapsed, 293.076s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.226s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 326.740s; elapsed, 83.028s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.226s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.430s; elapsed, 293.076s, #calls: 1', 'TOTAL : CPU, 620.670s; elapsed, 376.330s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 695.260s; elapsed, 88.480s', 'individual call time for EIGEN_LDLT: CPU, 306.330s; elapsed, 305.722s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.223s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 695.260s; elapsed, 88.480s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.223s, #calls: 1', 'time for EIGEN_LDLT: CPU, 306.330s; elapsed, 305.722s, #calls: 1', 'TOTAL : CPU, 1002.300s; elapsed, 394.425s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1519.880s; elapsed, 96.475s', 'individual call time for EIGEN_LDLT: CPU, 295.600s; elapsed, 293.198s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.040s; elapsed, 0.190s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1519.880s; elapsed, 96.475s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.040s; elapsed, 0.190s, #calls: 1', 'time for EIGEN_LDLT: CPU, 295.600s; elapsed, 293.198s, #calls: 1', 'TOTAL : CPU, 1816.520s; elapsed, 389.862s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.270s; elapsed, 150.371s', 'individual call time for EIGEN_LDLT: CPU, 370.420s; elapsed, 370.680s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.520s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.20204', '# - matching time = 0.145009', '# - symmetrization time = 0.100298', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.153986', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 140.958', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18415e+11, min = 41770, max = 7.18415e+11', '# - flop rate = 5.09666 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.20396e-10\trel.res = 2.87732e-15\tbw.error = 7.77224e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.278585', '# - total flops = 2.36199e+08, min = 177542, max = 2.36022e+08', '# - flop rate = 0.847853 GFlop/s', '# - bytes moved = 1221.56 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.38489 GByte/s', '# - solve arithmetic intensity = 0.193358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 142.910s; elapsed, 143.026s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 143.010s; elapsed, 143.012s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 142.910s; elapsed, 143.026s, #calls: 1', 'TOTAL : CPU, 142.910s; elapsed, 143.026s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.270s; elapsed, 150.371s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.520s, #calls: 1', 'time for EIGEN_LDLT: CPU, 370.420s; elapsed, 370.680s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 143.010s; elapsed, 143.012s, #calls: 1', 'TOTAL : CPU, 664.220s; elapsed, 664.582s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 209.060s; elapsed, 105.904s', 'individual call time for EIGEN_LDLT: CPU, 293.220s; elapsed, 293.217s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.248s', 'CPP -2', '# Initializing STRUMPACKCPP -2', '', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.14095', '# - matching time = 0.125592', '# - symmetrization time = 0.1009', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.157842', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 76.074', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 41770, max = 7.18416e+11', '# - flop rate = 9.44365 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1636e-10\trel.res = 2.78086e-15\tbw.error = 7.76343e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.298569', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.791419 GFlop/s', '# - bytes moved = 1223.99 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.09953 GByte/s', '# - solve arithmetic intensity = 0.193051 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 154.700s; elapsed, 78.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.180s; elapsed, 78.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 154.700s; elapsed, 78.070s, #calls: 1', 'TOTAL : CPU, 154.700s; elapsed, 78.070s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 209.060s; elapsed, 105.904s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.248s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.220s; elapsed, 293.217s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.180s; elapsed, 78.062s, #calls: 1', 'TOTAL : CPU, 581.830s; elapsed, 477.431s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 263.200s; elapsed, 67.225s', 'individual call time for EIGEN_LDLT: CPU, 293.070s; elapsed, 292.847s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.239s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.22281', '# - matching time = 0.145561', '# - symmetrization time = 0.104321', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.144889', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 66.8679', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 41770, max = 7.18416e+11', '# - flop rate = 10.7438 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15759e-10\trel.res = 2.7665e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.463599', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.509693 GFlop/s', '# - bytes moved = 1225.24 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.64289 GByte/s', '# - solve arithmetic intensity = 0.192855 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 268.890s; elapsed, 69.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.010s; elapsed, 69.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 268.890s; elapsed, 69.123s, #calls: 1', 'TOTAL : CPU, 268.890s; elapsed, 69.123s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 263.200s; elapsed, 67.225s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.070s; elapsed, 292.847s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.010s; elapsed, 69.120s, #calls: 1', 'TOTAL : CPU, 629.790s; elapsed, 429.431s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 506.160s; elapsed, 64.834s', 'individual call time for EIGEN_LDLT: CPU, 294.610s; elapsed, 293.457s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.198s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.16933', '# - matching time = 0.156434', '# - symmetrization time = 0.119043', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.143716', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 59.2749', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 42535, max = 7.18416e+11', '# - flop rate = 12.1201 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15744e-10\trel.res = 2.76614e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.365308', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.646833 GFlop/s', '# - bytes moved = 1225.31 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.35418 GByte/s', '# - solve arithmetic intensity = 0.192844 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 478.830s; elapsed, 61.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.660s; elapsed, 61.499s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 478.830s; elapsed, 61.506s, #calls: 1', 'TOTAL : CPU, 478.830s; elapsed, 61.506s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 506.160s; elapsed, 64.834s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.198s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.610s; elapsed, 293.457s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.660s; elapsed, 61.499s, #calls: 1', 'TOTAL : CPU, 872.050s; elapsed, 419.988s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1570.960s; elapsed, 99.793s', 'individual call time for EIGEN_LDLT: CPU, 296.150s; elapsed, 293.632s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.010s; elapsed, 0.195s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.18803', '# - matching time = 0.203366', '# - symmetrization time = 0.136586', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.168739', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 65.0366', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.1842e+11, min = 43208, max = 7.1842e+11', '# - flop rate = 11.0464 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15728e-10\trel.res = 2.76578e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.389212', '# - total flops = 2.36302e+08, min = 177542, max = 2.36125e+08', '# - flop rate = 0.60713 GFlop/s', '# - bytes moved = 1225.6 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.14893 GByte/s', '# - solve arithmetic intensity = 0.192806 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1053.250s; elapsed, 67.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 87.110s; elapsed, 67.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1053.250s; elapsed, 67.342s, #calls: 1', 'TOTAL : CPU, 1053.250s; elapsed, 67.342s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1570.960s; elapsed, 99.793s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.010s; elapsed, 0.195s, #calls: 1', 'time for EIGEN_LDLT: CPU, 296.150s; elapsed, 293.632s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 87.110s; elapsed, 67.334s, #calls: 1', 'TOTAL : CPU, 1955.230s; elapsed, 460.954s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 148.130s; elapsed, 148.247s', 'individual call time for EIGEN_LDLT: CPU, 297.780s; elapsed, 297.909s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.278s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.17228', '# - matching time = 0.129232', '# - symmetrization time = 0.0955942', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.102859', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 72.9503', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 9.8701 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10513e-10\trel.res = 2.64113e-15\tbw.error = 7.89018e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.219198', '# - total flops = 2.52082e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 1.15002 GFlop/s', '# - bytes moved = 460.028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09869 GByte/s', '# - solve arithmetic intensity = 0.547971 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 74.730s; elapsed, 74.789s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 74.700s; elapsed, 74.762s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 74.700s; elapsed, 74.756s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 74.780s; elapsed, 74.774s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 74.730s; elapsed, 74.789s, #calls: 1', 'TOTAL : CPU, 74.730s; elapsed, 74.789s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 74.700s; elapsed, 74.762s, #calls: 1', 'TOTAL : CPU, 74.700s; elapsed, 74.762s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 74.700s; elapsed, 74.756s, #calls: 1', 'TOTAL : CPU, 74.700s; elapsed, 74.756s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 148.130s; elapsed, 148.247s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.278s, #calls: 1', 'time for EIGEN_LDLT: CPU, 297.780s; elapsed, 297.909s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 74.780s; elapsed, 74.774s, #calls: 1', 'TOTAL : CPU, 520.970s; elapsed, 521.208s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 230.170s; elapsed, 118.565s', 'individual call time for EIGEN_LDLT: CPU, 314.690s; elapsed, 314.766s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.247s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.12244', '# - matching time = 0.132566', '# - symmetrization time = 0.09726', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.119828', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 62.0615', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 11.6018 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10477e-10\trel.res = 2.64028e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.206869', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 1.21864 GFlop/s', '# - bytes moved = 460.383 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.22548 GByte/s', '# - solve arithmetic intensity = 0.547585 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 92.820s; elapsed, 63.858s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 124.700s; elapsed, 63.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 116.590s; elapsed, 63.825s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.790s; elapsed, 63.844s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 116.590s; elapsed, 63.825s, #calls: 1', 'TOTAL : CPU, 116.590s; elapsed, 63.825s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 92.820s; elapsed, 63.858s, #calls: 1', 'TOTAL : CPU, 92.820s; elapsed, 63.858s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 124.700s; elapsed, 63.834s, #calls: 1', 'TOTAL : CPU, 124.700s; elapsed, 63.834s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 230.170s; elapsed, 118.565s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.247s, #calls: 1', 'time for EIGEN_LDLT: CPU, 314.690s; elapsed, 314.766s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.790s; elapsed, 63.844s, #calls: 1', 'TOTAL : CPU, 610.030s; elapsed, 497.423s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 267.810s; elapsed, 69.161s', 'individual call time for EIGEN_LDLT: CPU, 310.330s; elapsed, 310.086s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.259s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.15956', '# - matching time = 0.158444', '# - symmetrization time = 0.107268', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.135009', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 61.1852', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 11.768 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1046e-10\trel.res = 2.63986e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.231617', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 1.08843 GFlop/s', '# - bytes moved = 460.395 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98774 GByte/s', '# - solve arithmetic intensity = 0.54757 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 142.980s; elapsed, 63.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.670s; elapsed, 63.089s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 203.220s; elapsed, 63.084s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 66.190s; elapsed, 63.099s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.670s; elapsed, 63.089s, #calls: 1', 'TOTAL : CPU, 243.670s; elapsed, 63.089s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 203.220s; elapsed, 63.084s, #calls: 1', 'TOTAL : CPU, 203.220s; elapsed, 63.084s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 142.980s; elapsed, 63.114s, #calls: 1', 'TOTAL : CPU, 142.980s; elapsed, 63.114s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 267.810s; elapsed, 69.161s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.259s, #calls: 1', 'time for EIGEN_LDLT: CPU, 310.330s; elapsed, 310.086s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 66.190s; elapsed, 63.099s, #calls: 1', 'TOTAL : CPU, 644.880s; elapsed, 442.605s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 783.020s; elapsed, 99.964s', 'individual call time for EIGEN_LDLT: CPU, 298.390s; elapsed, 297.408s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.650s; elapsed, 0.210s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '1.18529', '# - matching time = 0.152289', '# - symmetrization time = 0.109042', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.134842', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 69.0459', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 42535, max = 5.52422e+11', '# - flop rate = 10.4282 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10436e-10\trel.res = 2.6393e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.330425', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 0.762953 GFlop/s', '# - bytes moved = 460.421 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.39342 GByte/s', '# - solve arithmetic intensity = 0.54754 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.990s; elapsed, 71.110s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 463.060s; elapsed, 71.078s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 540.750s; elapsed, 71.086s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.020s; elapsed, 71.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 463.060s; elapsed, 71.078s, #calls: 1', 'TOTAL : CPU, 463.060s; elapsed, 71.078s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 540.750s; elapsed, 71.086s, #calls: 1', 'TOTAL : CPU, 540.750s; elapsed, 71.086s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.990s; elapsed, 71.110s, #calls: 1', 'TOTAL : CPU, 310.990s; elapsed, 71.110s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 783.020s; elapsed, 99.964s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.650s; elapsed, 0.210s, #calls: 1', 'time for EIGEN_LDLT: CPU, 298.390s; elapsed, 297.408s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.020s; elapsed, 71.097s, #calls: 1', 'TOTAL : CPU, 1161.080s; elapsed, 468.679s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.690s; elapsed, 150.815s', 'individual call time for EIGEN_LDLT: CPU, 302.900s; elapsed, 302.983s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.285s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.20141', '# - matching time = 0.13265', '# - symmetrization time = 0.0661378', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.114449', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 71.1269', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 10.1196 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.24362e-11\trel.res = 1.97013e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.197882', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.45578 GFlop/s', '# - bytes moved = 434.79 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.19721 GByte/s', '# - solve arithmetic intensity = 0.662556 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 72.850s; elapsed, 72.930s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.900s; elapsed, 72.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 72.870s; elapsed, 72.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.900s; elapsed, 72.931s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 72.830s; elapsed, 72.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 72.840s; elapsed, 72.945s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 72.900s; elapsed, 72.945s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.930s; elapsed, 72.941s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 72.850s; elapsed, 72.930s, #calls: 1', 'TOTAL : CPU, 72.850s; elapsed, 72.930s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 72.870s; elapsed, 72.926s, #calls: 1', 'TOTAL : CPU, 72.870s; elapsed, 72.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.900s; elapsed, 72.928s, #calls: 1', 'TOTAL : CPU, 72.900s; elapsed, 72.928s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 72.900s; elapsed, 72.945s, #calls: 1', 'TOTAL : CPU, 72.900s; elapsed, 72.945s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 72.830s; elapsed, 72.923s, #calls: 1', 'TOTAL : CPU, 72.830s; elapsed, 72.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.900s; elapsed, 72.931s, #calls: 1', 'TOTAL : CPU, 72.900s; elapsed, 72.931s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 72.840s; elapsed, 72.945s, #calls: 1', 'TOTAL : CPU, 72.840s; elapsed, 72.945s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.690s; elapsed, 150.815s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.285s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.900s; elapsed, 302.983s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.930s; elapsed, 72.941s, #calls: 1', 'TOTAL : CPU, 526.810s; elapsed, 527.025s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 214.450s; elapsed, 108.659s', 'individual call time for EIGEN_LDLT: CPU, 314.730s; elapsed, 314.765s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.254s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# - nd time = 1.16882', '# - matching time = 0.135947', '# - symmetrization time = 0.0658722', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.121035', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 60.7569', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 11.8468 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.22963e-11\trel.res = 1.96679e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.149498', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.92693 GFlop/s', '# - bytes moved = 434.803 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.90842 GByte/s', '# - solve arithmetic intensity = 0.662536 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 81.040s; elapsed, 62.498s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.870s; elapsed, 62.486s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.730s; elapsed, 62.487s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.370s; elapsed, 62.483s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 74.170s; elapsed, 62.484s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 110.850s; elapsed, 62.481s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.950s; elapsed, 62.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 63.380s; elapsed, 62.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.870s; elapsed, 62.486s, #calls: 1', 'TOTAL : CPU, 120.870s; elapsed, 62.486s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.730s; elapsed, 62.487s, #calls: 1', 'TOTAL : CPU, 72.730s; elapsed, 62.487s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 110.850s; elapsed, 62.481s, #calls: 1', 'TOTAL : CPU, 110.850s; elapsed, 62.481s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.370s; elapsed, 62.483s, #calls: 1', 'TOTAL : CPU, 72.370s; elapsed, 62.483s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 74.170s; elapsed, 62.484s, #calls: 1', 'TOTAL : CPU, 74.170s; elapsed, 62.484s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.950s; elapsed, 62.500s, #calls: 1', 'TOTAL : CPU, 86.950s; elapsed, 62.500s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 81.040s; elapsed, 62.498s, #calls: 1', 'TOTAL : CPU, 81.040s; elapsed, 62.498s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 214.450s; elapsed, 108.659s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.254s, #calls: 1', 'time for EIGEN_LDLT: CPU, 314.730s; elapsed, 314.765s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 63.380s; elapsed, 62.496s, #calls: 1', 'TOTAL : CPU, 592.940s; elapsed, 486.174s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 258.760s; elapsed, 66.629s', 'individual call time for EIGEN_LDLT: CPU, 308.160s; elapsed, 307.768s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.220s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.22428', '# - matching time = 0.153419', '# - symmetrization time = 0.0756512', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.120191', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 65.8296', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 10.9339 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.22801e-11\trel.res = 1.9664e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.212694', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.3544 GFlop/s', '# - bytes moved = 434.821 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04435 GByte/s', '# - solve arithmetic intensity = 0.662508 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 118.050s; elapsed, 67.735s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 156.850s; elapsed, 67.736s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 94.270s; elapsed, 67.722s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 235.640s; elapsed, 67.716s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 99.970s; elapsed, 67.719s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 95.900s; elapsed, 67.721s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 256.220s; elapsed, 67.720s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.640s; elapsed, 67.732s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 235.640s; elapsed, 67.716s, #calls: 1', 'TOTAL : CPU, 235.640s; elapsed, 67.716s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 118.050s; elapsed, 67.735s, #calls: 1', 'TOTAL : CPU, 118.050s; elapsed, 67.735s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 256.220s; elapsed, 67.720s, #calls: 1', 'TOTAL : CPU, 256.220s; elapsed, 67.720s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 156.850s; elapsed, 67.736s, #calls: 1', 'TOTAL : CPU, 156.850s; elapsed, 67.736s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 95.900s; elapsed, 67.721s, #calls: 1', 'TOTAL : CPU, 95.900s; elapsed, 67.721s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 99.970s; elapsed, 67.719s, #calls: 1', 'TOTAL : CPU, 99.970s; elapsed, 67.719s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 94.270s; elapsed, 67.722s, #calls: 1', 'TOTAL : CPU, 94.270s; elapsed, 67.722s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 258.760s; elapsed, 66.629s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.220s, #calls: 1', 'time for EIGEN_LDLT: CPU, 308.160s; elapsed, 307.768s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.640s; elapsed, 67.732s, #calls: 1', 'TOTAL : CPU, 638.050s; elapsed, 442.348s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 159.620s; elapsed, 159.742s', 'individual call time for EIGEN_LDLT: CPU, 310.950s; elapsed, 311.139s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.307s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 127', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.29305', '# - matching time = 0.145027', '# - symmetrization time = 0.0552728', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.128587', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 60.253', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51438e+11', '# - flop rate = 11.9458 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.27001e+11', '# --------------------------------------------', '# total = 7.27001e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.95624e-11\trel.res = 2.14044e-15\tbw.error = 6.17961e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.141461', '# - total flops = 3.64315e+08, min = 100280, max = 1.87793e+08', '# - flop rate = 2.57538 GFlop/s', '# - bytes moved = 412.812 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.91821 GByte/s', '# - solve arithmetic intensity = 0.882521 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 61.980s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.990s; elapsed, 62.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 62.040s; elapsed, 62.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 61.910s; elapsed, 62.107s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 62.050s; elapsed, 62.105s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 62.010s; elapsed, 62.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 62.000s; elapsed, 62.107s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 61.970s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 62.010s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 62.060s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 62.040s; elapsed, 62.096s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 62.050s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 62.020s; elapsed, 62.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 62.010s; elapsed, 62.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 61.950s; elapsed, 62.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 62.080s; elapsed, 62.108s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 62.010s; elapsed, 62.109s, #calls: 1', 'TOTAL : CPU, 62.010s; elapsed, 62.109s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 61.970s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 61.970s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 62.010s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 62.010s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 62.040s; elapsed, 62.097s, #calls: 1', 'TOTAL : CPU, 62.040s; elapsed, 62.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 62.020s; elapsed, 62.097s, #calls: 1', 'TOTAL : CPU, 62.020s; elapsed, 62.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 61.980s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 61.980s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 61.950s; elapsed, 62.097s, #calls: 1', 'TOTAL : CPU, 61.950s; elapsed, 62.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 62.040s; elapsed, 62.096s, #calls: 1', 'TOTAL : CPU, 62.040s; elapsed, 62.096s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 61.910s; elapsed, 62.107s, #calls: 1', 'TOTAL : CPU, 61.910s; elapsed, 62.107s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 62.050s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 62.050s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 62.010s; elapsed, 62.100s, #calls: 1', 'TOTAL : CPU, 62.010s; elapsed, 62.100s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 62.060s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 62.060s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 62.050s; elapsed, 62.105s, #calls: 1', 'TOTAL : CPU, 62.050s; elapsed, 62.105s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.990s; elapsed, 62.100s, #calls: 1', 'TOTAL : CPU, 61.990s; elapsed, 62.100s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 62.000s; elapsed, 62.107s, #calls: 1', 'TOTAL : CPU, 62.000s; elapsed, 62.107s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 159.620s; elapsed, 159.742s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.307s, #calls: 1', 'time for EIGEN_LDLT: CPU, 310.950s; elapsed, 311.139s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 62.080s; elapsed, 62.108s, #calls: 1', 'TOTAL : CPU, 532.950s; elapsed, 533.296s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 175.680s; elapsed, 88.581s', 'individual call time for EIGEN_LDLT: CPU, 365.500s; elapsed, 365.574s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.346s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 127', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.27181', '# - matching time = 0.166307', '# - symmetrization time = 0.0745802', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.138116', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 86.6161', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51438e+11', '# - flop rate = 8.30993 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.27001e+11', '# --------------------------------------------', '# total = 7.27001e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.94628e-11\trel.res = 2.13806e-15\tbw.error = 6.1844e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.135193', '# - total flops = 3.64315e+08, min = 100280, max = 1.87793e+08', '# - flop rate = 2.69478 GFlop/s', '# - bytes moved = 412.822 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.05358 GByte/s', '# - solve arithmetic intensity = 0.882499 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 97.020s; elapsed, 88.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 103.890s; elapsed, 88.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 103.010s; elapsed, 88.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 96.290s; elapsed, 88.497s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 115.400s; elapsed, 88.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 96.010s; elapsed, 88.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 170.210s; elapsed, 88.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 94.980s; elapsed, 88.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 94.550s; elapsed, 88.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 97.080s; elapsed, 88.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 96.670s; elapsed, 88.501s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 95.880s; elapsed, 88.499s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 94.990s; elapsed, 88.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 136.260s; elapsed, 88.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 96.680s; elapsed, 88.495s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 89.420s; elapsed, 88.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 94.550s; elapsed, 88.496s, #calls: 1', 'TOTAL : CPU, 94.550s; elapsed, 88.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 96.290s; elapsed, 88.497s, #calls: 1', 'TOTAL : CPU, 96.290s; elapsed, 88.497s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 97.080s; elapsed, 88.496s, #calls: 1', 'TOTAL : CPU, 97.080s; elapsed, 88.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 96.010s; elapsed, 88.502s, #calls: 1', 'TOTAL : CPU, 96.010s; elapsed, 88.502s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 95.880s; elapsed, 88.499s, #calls: 1', 'TOTAL : CPU, 95.880s; elapsed, 88.499s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 170.210s; elapsed, 88.500s, #calls: 1', 'TOTAL : CPU, 170.210s; elapsed, 88.500s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 94.990s; elapsed, 88.502s, #calls: 1', 'TOTAL : CPU, 94.990s; elapsed, 88.502s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 97.020s; elapsed, 88.505s, #calls: 1', 'TOTAL : CPU, 97.020s; elapsed, 88.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 103.010s; elapsed, 88.504s, #calls: 1', 'TOTAL : CPU, 103.010s; elapsed, 88.504s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 96.680s; elapsed, 88.495s, #calls: 1', 'TOTAL : CPU, 96.680s; elapsed, 88.495s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 103.890s; elapsed, 88.506s, #calls: 1', 'TOTAL : CPU, 103.890s; elapsed, 88.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 115.400s; elapsed, 88.506s, #calls: 1', 'TOTAL : CPU, 115.400s; elapsed, 88.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 136.260s; elapsed, 88.494s, #calls: 1', 'TOTAL : CPU, 136.260s; elapsed, 88.494s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 94.980s; elapsed, 88.500s, #calls: 1', 'TOTAL : CPU, 94.980s; elapsed, 88.500s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 96.670s; elapsed, 88.501s, #calls: 1', 'TOTAL : CPU, 96.670s; elapsed, 88.501s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 175.680s; elapsed, 88.581s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.346s, #calls: 1', 'time for EIGEN_LDLT: CPU, 365.500s; elapsed, 365.574s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 89.420s; elapsed, 88.511s, #calls: 1', 'TOTAL : CPU, 631.100s; elapsed, 543.012s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.350s; elapsed, 31.372s', 'individual call time for EIGEN_LDLT: CPU, 37.620s; elapsed, 37.649s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.120s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.350s; elapsed, 31.372s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.620s; elapsed, 37.649s, #calls: 1', 'TOTAL : CPU, 69.100s; elapsed, 69.141s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.630s; elapsed, 20.578s', 'individual call time for EIGEN_LDLT: CPU, 37.840s; elapsed, 37.683s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.630s; elapsed, 20.578s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.840s; elapsed, 37.683s, #calls: 1', 'TOTAL : CPU, 78.640s; elapsed, 58.372s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 67.890s; elapsed, 17.197s', 'individual call time for EIGEN_LDLT: CPU, 40.040s; elapsed, 39.527s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.102s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 67.890s; elapsed, 17.197s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.040s; elapsed, 39.527s, #calls: 1', 'TOTAL : CPU, 108.140s; elapsed, 56.825s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 119.830s; elapsed, 15.314s', 'individual call time for EIGEN_LDLT: CPU, 38.900s; elapsed, 37.665s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.119s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 119.830s; elapsed, 15.314s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.900s; elapsed, 37.665s, #calls: 1', 'TOTAL : CPU, 159.110s; elapsed, 53.098s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 293.910s; elapsed, 18.708s', 'individual call time for EIGEN_LDLT: CPU, 40.390s; elapsed, 37.670s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.089s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 293.910s; elapsed, 18.708s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.089s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.390s; elapsed, 37.670s, #calls: 1', 'TOTAL : CPU, 334.770s; elapsed, 56.467s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.770s; elapsed, 32.792s', 'individual call time for EIGEN_LDLT: CPU, 38.960s; elapsed, 38.994s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.124s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.4046', '# - matching time = 0.0722611', '# - symmetrization time = 0.0358469', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.085063', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 31.6957', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48564e+10, min = 222880, max = 9.48562e+10', '# - flop rate = 2.99272 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.11914e-11\trel.res = 2.20224e-15\tbw.error = 4.90384e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120098', '# - total flops = 6.78544e+07, min = 289862, max = 6.75645e+07', '# - flop rate = 0.564992 GFlop/s', '# - bytes moved = 410.995 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.42216 GByte/s', '# - solve arithmetic intensity = 0.165098 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.490s; elapsed, 32.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.510s; elapsed, 32.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.490s; elapsed, 32.511s, #calls: 1', 'TOTAL : CPU, 32.490s; elapsed, 32.511s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.770s; elapsed, 32.792s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.124s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.960s; elapsed, 38.994s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.510s; elapsed, 32.505s, #calls: 1', 'TOTAL : CPU, 104.360s; elapsed, 104.415s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.770s; elapsed, 18.630s', 'individual call time for EIGEN_LDLT: CPU, 39.070s; elapsed, 38.922s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.121s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.380367', '# - matching time = 0.0759459', '# - symmetrization time = 0.0353429', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0845001', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 19.3656', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 222880, max = 9.48565e+10', '# - flop rate = 4.8982 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04299e-11\trel.res = 2.18385e-15\tbw.error = 4.87162e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0978129', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.693914 GFlop/s', '# - bytes moved = 411.788 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.20996 GByte/s', '# - solve arithmetic intensity = 0.164827 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 39.960s; elapsed, 20.144s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.860s; elapsed, 20.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 39.960s; elapsed, 20.144s, #calls: 1', 'TOTAL : CPU, 39.960s; elapsed, 20.144s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.770s; elapsed, 18.630s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.070s; elapsed, 38.922s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.860s; elapsed, 20.133s, #calls: 1', 'TOTAL : CPU, 96.880s; elapsed, 77.805s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 59.180s; elapsed, 15.084s', 'individual call time for EIGEN_LDLT: CPU, 69.840s; elapsed, 69.335s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.377711', '# - matching time = 0.0720251', '# - symmetrization time = 0.035423', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.080492', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 16.0621', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 222880, max = 9.48565e+10', '# - flop rate = 5.90561 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04185e-11\trel.res = 2.18357e-15\tbw.error = 4.87162e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.111008', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.61143 GFlop/s', '# - bytes moved = 411.808 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.70971 GByte/s', '# - solve arithmetic intensity = 0.164819 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.460s; elapsed, 16.843s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.070s; elapsed, 16.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.460s; elapsed, 16.843s, #calls: 1', 'TOTAL : CPU, 66.460s; elapsed, 16.843s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 59.180s; elapsed, 15.084s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 69.840s; elapsed, 69.335s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.070s; elapsed, 16.838s, #calls: 1', 'TOTAL : CPU, 148.290s; elapsed, 101.362s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 116.070s; elapsed, 14.850s', 'individual call time for EIGEN_LDLT: CPU, 40.230s; elapsed, 39.012s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.088s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.408596', '# - matching time = 0.0735049', '# - symmetrization time = 0.0369918', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0991609', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 15.4322', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 224830, max = 9.48565e+10', '# - flop rate = 6.14666 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04226e-11\trel.res = 2.18367e-15\tbw.error = 4.86687e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.161948', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.419108 GFlop/s', '# - bytes moved = 411.848 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.54309 GByte/s', '# - solve arithmetic intensity = 0.164803 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.390s; elapsed, 16.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.000s; elapsed, 16.312s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.390s; elapsed, 16.319s, #calls: 1', 'TOTAL : CPU, 128.390s; elapsed, 16.319s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 116.070s; elapsed, 14.850s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.230s; elapsed, 39.012s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.000s; elapsed, 16.312s, #calls: 1', 'TOTAL : CPU, 178.570s; elapsed, 70.262s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 253.890s; elapsed, 16.229s', 'individual call time for EIGEN_LDLT: CPU, 40.450s; elapsed, 37.825s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.095s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.423476', '# - matching time = 0.109122', '# - symmetrization time = 0.0572441', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.104183', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 14.5253', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48577e+10, min = 226855, max = 9.48575e+10', '# - flop rate = 6.53052 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04391e-11\trel.res = 2.18407e-15\tbw.error = 4.87578e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.126633', '# - total flops = 6.78784e+07, min = 289862, max = 6.75885e+07', '# - flop rate = 0.536025 GFlop/s', '# - bytes moved = 412.003 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.25352 GByte/s', '# - solve arithmetic intensity = 0.164752 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 241.990s; elapsed, 15.480s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.120s; elapsed, 15.471s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 241.990s; elapsed, 15.480s, #calls: 1', 'TOTAL : CPU, 241.990s; elapsed, 15.480s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 253.890s; elapsed, 16.229s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.095s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.450s; elapsed, 37.825s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.120s; elapsed, 15.471s, #calls: 1', 'TOTAL : CPU, 322.980s; elapsed, 69.620s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 34.813s', 'individual call time for EIGEN_LDLT: CPU, 39.810s; elapsed, 39.832s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.136s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.54868', '# - matching time = 0.0796881', '# - symmetrization time = 0.0392759', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0620191', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 15.6672', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 6.02978 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00117e-10\trel.res = 2.4178e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.093749', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.763024 GFlop/s', '# - bytes moved = 219.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.34177 GByte/s', '# - solve arithmetic intensity = 0.325832 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.530s; elapsed, 16.547s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.510s; elapsed, 16.534s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.500s; elapsed, 16.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.540s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.530s; elapsed, 16.547s, #calls: 1', 'TOTAL : CPU, 16.530s; elapsed, 16.547s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.500s; elapsed, 16.531s, #calls: 1', 'TOTAL : CPU, 16.500s; elapsed, 16.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.510s; elapsed, 16.534s, #calls: 1', 'TOTAL : CPU, 16.510s; elapsed, 16.534s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 34.813s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.136s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.810s; elapsed, 39.832s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.540s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 91.270s; elapsed, 91.324s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 41.140s; elapsed, 20.889s', 'individual call time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.798s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.459112', '# - matching time = 0.0695028', '# - symmetrization time = 0.0358191', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.064703', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 14.3552', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 6.58089 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.99927e-11\trel.res = 2.41479e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0859909', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.831865 GFlop/s', '# - bytes moved = 219.551 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55319 GByte/s', '# - solve arithmetic intensity = 0.325814 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.320s; elapsed, 15.134s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.830s; elapsed, 15.124s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.510s; elapsed, 15.118s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.790s; elapsed, 15.131s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.320s; elapsed, 15.134s, #calls: 1', 'TOTAL : CPU, 23.320s; elapsed, 15.134s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.830s; elapsed, 15.124s, #calls: 1', 'TOTAL : CPU, 25.830s; elapsed, 15.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.510s; elapsed, 15.118s, #calls: 1', 'TOTAL : CPU, 29.510s; elapsed, 15.118s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 41.140s; elapsed, 20.889s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.798s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.790s; elapsed, 15.131s, #calls: 1', 'TOTAL : CPU, 95.050s; elapsed, 73.933s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 68.560s; elapsed, 17.442s', 'individual call time for EIGEN_LDLT: CPU, 40.820s; elapsed, 40.320s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.505306', '# - matching time = 0.0820031', '# - symmetrization time = 0.0443912', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.067661', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 15.0046', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 6.29607 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.99807e-11\trel.res = 2.4145e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0940771', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.760363 GFlop/s', '# - bytes moved = 219.566 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33389 GByte/s', '# - solve arithmetic intensity = 0.325792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 45.130s; elapsed, 15.858s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 33.900s; elapsed, 15.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.460s; elapsed, 15.852s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.920s; elapsed, 15.864s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.460s; elapsed, 15.852s, #calls: 1', 'TOTAL : CPU, 61.460s; elapsed, 15.852s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 45.130s; elapsed, 15.858s, #calls: 1', 'TOTAL : CPU, 45.130s; elapsed, 15.858s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 33.900s; elapsed, 15.869s, #calls: 1', 'TOTAL : CPU, 33.900s; elapsed, 15.869s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 68.560s; elapsed, 17.442s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.820s; elapsed, 40.320s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.920s; elapsed, 15.864s, #calls: 1', 'TOTAL : CPU, 127.510s; elapsed, 73.742s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 113.430s; elapsed, 14.489s', 'individual call time for EIGEN_LDLT: CPU, 40.390s; elapsed, 39.134s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.104s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.503232', '# - matching time = 0.0831001', '# - symmetrization time = 0.0405271', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0808351', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 15.9388', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 224964, max = 7.03507e+10', '# - flop rate = 5.92703 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9961e-11\trel.res = 2.41402e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0939651', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.76127 GFlop/s', '# - bytes moved = 219.596 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33699 GByte/s', '# - solve arithmetic intensity = 0.325748 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 80.390s; elapsed, 16.805s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.690s; elapsed, 16.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 129.540s; elapsed, 16.799s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.830s; elapsed, 16.812s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 80.390s; elapsed, 16.805s, #calls: 1', 'TOTAL : CPU, 80.390s; elapsed, 16.805s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 129.540s; elapsed, 16.799s, #calls: 1', 'TOTAL : CPU, 129.540s; elapsed, 16.799s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.690s; elapsed, 16.815s, #calls: 1', 'TOTAL : CPU, 60.690s; elapsed, 16.815s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 113.430s; elapsed, 14.489s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.390s; elapsed, 39.134s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.830s; elapsed, 16.812s, #calls: 1', 'TOTAL : CPU, 175.940s; elapsed, 70.538s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.240s; elapsed, 33.242s', 'individual call time for EIGEN_LDLT: CPU, 39.450s; elapsed, 39.459s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.127s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.499541', '# - matching time = 0.0728931', '# - symmetrization time = 0.026294', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0576689', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 15.3904', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 6.18427 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01808e-10\trel.res = 2.45864e-15\tbw.error = 4.93262e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.078059', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 1.04269 GFlop/s', '# - bytes moved = 212.161 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71795 GByte/s', '# - solve arithmetic intensity = 0.38363 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.160s; elapsed, 16.180s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.140s; elapsed, 16.178s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.160s; elapsed, 16.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.130s; elapsed, 16.168s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.130s; elapsed, 16.173s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.160s; elapsed, 16.167s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.160s; elapsed, 16.173s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.180s; elapsed, 16.180s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.160s; elapsed, 16.180s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.180s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.160s; elapsed, 16.179s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.130s; elapsed, 16.173s, #calls: 1', 'TOTAL : CPU, 16.130s; elapsed, 16.173s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.130s; elapsed, 16.168s, #calls: 1', 'TOTAL : CPU, 16.130s; elapsed, 16.168s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.160s; elapsed, 16.167s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.167s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.140s; elapsed, 16.178s, #calls: 1', 'TOTAL : CPU, 16.140s; elapsed, 16.178s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.160s; elapsed, 16.173s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.173s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.240s; elapsed, 33.242s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.127s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.450s; elapsed, 39.459s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.180s; elapsed, 16.180s, #calls: 1', 'TOTAL : CPU, 89.000s; elapsed, 89.007s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.510s; elapsed, 18.470s', 'individual call time for EIGEN_LDLT: CPU, 39.030s; elapsed, 38.883s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.486414', '# - matching time = 0.0734921', '# - symmetrization time = 0.0299671', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0651522', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 13.9594', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 6.81821 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01772e-10\trel.res = 2.45777e-15\tbw.error = 4.93262e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.077909', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 1.04469 GFlop/s', '# - bytes moved = 212.169 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.72329 GByte/s', '# - solve arithmetic intensity = 0.383615 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.130s; elapsed, 14.748s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.570s; elapsed, 14.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.550s; elapsed, 14.742s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.840s; elapsed, 14.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.680s; elapsed, 14.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.750s; elapsed, 14.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 14.743s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.400s; elapsed, 14.750s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.550s; elapsed, 14.742s, #calls: 1', 'TOTAL : CPU, 28.550s; elapsed, 14.742s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.130s; elapsed, 14.748s, #calls: 1', 'TOTAL : CPU, 17.130s; elapsed, 14.748s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.570s; elapsed, 14.749s, #calls: 1', 'TOTAL : CPU, 20.570s; elapsed, 14.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.680s; elapsed, 14.737s, #calls: 1', 'TOTAL : CPU, 16.680s; elapsed, 14.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.840s; elapsed, 14.749s, #calls: 1', 'TOTAL : CPU, 18.840s; elapsed, 14.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.750s; elapsed, 14.737s, #calls: 1', 'TOTAL : CPU, 24.750s; elapsed, 14.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 14.743s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 14.743s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.510s; elapsed, 18.470s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.030s; elapsed, 38.883s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.400s; elapsed, 14.750s, #calls: 1', 'TOTAL : CPU, 91.090s; elapsed, 72.211s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 63.810s; elapsed, 16.253s', 'individual call time for EIGEN_LDLT: CPU, 40.720s; elapsed, 40.197s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.56894', '# - matching time = 0.0841079', '# - symmetrization time = 0.0454419', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.072114', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 15.1333', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 6.28933 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01768e-10\trel.res = 2.45766e-15\tbw.error = 4.92775e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0940831', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 0.865098 GFlop/s', '# - bytes moved = 212.179 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25523 GByte/s', '# - solve arithmetic intensity = 0.383597 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.620s; elapsed, 16.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.660s; elapsed, 16.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.900s; elapsed, 16.060s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 43.880s; elapsed, 16.048s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.290s; elapsed, 16.054s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.700s; elapsed, 16.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.890s; elapsed, 16.047s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.150s; elapsed, 16.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.700s; elapsed, 16.057s, #calls: 1', 'TOTAL : CPU, 22.700s; elapsed, 16.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.620s; elapsed, 16.061s, #calls: 1', 'TOTAL : CPU, 23.620s; elapsed, 16.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 43.880s; elapsed, 16.048s, #calls: 1', 'TOTAL : CPU, 43.880s; elapsed, 16.048s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.290s; elapsed, 16.054s, #calls: 1', 'TOTAL : CPU, 61.290s; elapsed, 16.054s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.660s; elapsed, 16.062s, #calls: 1', 'TOTAL : CPU, 35.660s; elapsed, 16.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.890s; elapsed, 16.047s, #calls: 1', 'TOTAL : CPU, 21.890s; elapsed, 16.047s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.900s; elapsed, 16.060s, #calls: 1', 'TOTAL : CPU, 27.900s; elapsed, 16.060s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 63.810s; elapsed, 16.253s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.720s; elapsed, 40.197s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.150s; elapsed, 16.061s, #calls: 1', 'TOTAL : CPU, 122.900s; elapsed, 72.621s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 37.820s; elapsed, 37.893s', 'individual call time for EIGEN_LDLT: CPU, 38.850s; elapsed, 38.878s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.126s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,191', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.503834', '# - matching time = 0.0723329', '# - symmetrization time = 0.024868', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,191', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.069247', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.38 MB', '# - factor time = 16.3407', '# - factor nonzeros = 31,297,498', '# - factor memory = 250.38 MB', '# - total flops = 9.44704e+10, min = 223014, max = 7.0108e+10', '# - flop rate = 5.78128 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.62837e+10', '# --------------------------------------------', '# total = 9.62837e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.88967e-11\trel.res = 1.66383e-15\tbw.error = 3.71324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079927', '# - total flops = 1.00397e+08, min = 114336, max = 5.02255e+07', '# - flop rate = 1.25612 GFlop/s', '# - bytes moved = 204.061 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55309 GByte/s', '# - solve arithmetic intensity = 0.491997 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.100s; elapsed, 17.139s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.110s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.090s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.080s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.120s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.120s; elapsed, 17.138s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.100s; elapsed, 17.138s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.110s; elapsed, 17.132s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.080s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.090s; elapsed, 17.140s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 17.137s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.080s; elapsed, 17.137s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.110s; elapsed, 17.138s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.070s; elapsed, 17.140s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.070s; elapsed, 17.131s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.150s; elapsed, 17.145s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.070s; elapsed, 17.140s, #calls: 1', 'TOTAL : CPU, 17.070s; elapsed, 17.140s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.080s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.080s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.120s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.120s; elapsed, 17.138s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 17.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.110s; elapsed, 17.138s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.110s; elapsed, 17.132s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.132s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.080s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.080s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.100s; elapsed, 17.138s, #calls: 1', 'TOTAL : CPU, 17.100s; elapsed, 17.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.090s; elapsed, 17.140s, #calls: 1', 'TOTAL : CPU, 17.090s; elapsed, 17.140s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.100s; elapsed, 17.139s, #calls: 1', 'TOTAL : CPU, 17.100s; elapsed, 17.139s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.090s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.090s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.110s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.070s; elapsed, 17.131s, #calls: 1', 'TOTAL : CPU, 17.070s; elapsed, 17.131s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.080s; elapsed, 17.137s, #calls: 1', 'TOTAL : CPU, 17.080s; elapsed, 17.137s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 17.137s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 17.137s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 37.820s; elapsed, 37.893s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.126s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.850s; elapsed, 38.878s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.150s; elapsed, 17.145s, #calls: 1', 'TOTAL : CPU, 93.950s; elapsed, 94.041s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.080s; elapsed, 19.805s', 'individual call time for EIGEN_LDLT: CPU, 40.030s; elapsed, 39.881s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,191', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.567631', '# - matching time = 0.0831571', '# - symmetrization time = 0.027292', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,191', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0821381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.38 MB', '# - factor time = 14.1487', '# - factor nonzeros = 31,297,498', '# - factor memory = 250.38 MB', '# - total flops = 9.44704e+10, min = 223014, max = 7.0108e+10', '# - flop rate = 6.67699 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.62837e+10', '# --------------------------------------------', '# total = 9.62837e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.88907e-11\trel.res = 1.66368e-15\tbw.error = 3.71324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.082845', '# - total flops = 1.00397e+08, min = 114336, max = 5.02255e+07', '# - flop rate = 1.21187 GFlop/s', '# - bytes moved = 204.065 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.46321 GByte/s', '# - solve arithmetic intensity = 0.491988 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.710s; elapsed, 15.051s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.120s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.580s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 19.200s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.350s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 19.170s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.700s; elapsed, 15.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.420s; elapsed, 15.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.220s; elapsed, 15.053s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.260s; elapsed, 15.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.150s; elapsed, 15.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.770s; elapsed, 15.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.790s; elapsed, 15.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.640s; elapsed, 15.042s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 23.950s; elapsed, 15.044s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.740s; elapsed, 15.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.710s; elapsed, 15.051s, #calls: 1', 'TOTAL : CPU, 17.710s; elapsed, 15.051s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.700s; elapsed, 15.049s, #calls: 1', 'TOTAL : CPU, 17.700s; elapsed, 15.049s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.580s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 17.580s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.120s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.420s; elapsed, 15.049s, #calls: 1', 'TOTAL : CPU, 17.420s; elapsed, 15.049s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.150s; elapsed, 15.049s, #calls: 1', 'TOTAL : CPU, 17.150s; elapsed, 15.049s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 19.200s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 19.200s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 23.950s; elapsed, 15.044s, #calls: 1', 'TOTAL : CPU, 23.950s; elapsed, 15.044s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.350s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 17.350s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.260s; elapsed, 15.050s, #calls: 1', 'TOTAL : CPU, 29.260s; elapsed, 15.050s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.640s; elapsed, 15.042s, #calls: 1', 'TOTAL : CPU, 16.640s; elapsed, 15.042s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.790s; elapsed, 15.050s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.770s; elapsed, 15.050s, #calls: 1', 'TOTAL : CPU, 16.790s; elapsed, 15.050s', 'TOTAL : CPU, 16.770s; elapsed, 15.050s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 19.170s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 19.170s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.220s; elapsed, 15.053s, #calls: 1', 'TOTAL : CPU, 21.220s; elapsed, 15.053s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.080s; elapsed, 19.805s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.030s; elapsed, 39.881s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.740s; elapsed, 15.057s, #calls: 1', 'TOTAL : CPU, 95.010s; elapsed, 74.859s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.060s; elapsed, 31.085s', 'individual call time for EIGEN_LDLT: CPU, 37.640s; elapsed, 37.659s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.060s; elapsed, 31.085s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.640s; elapsed, 37.659s, #calls: 1', 'TOTAL : CPU, 68.810s; elapsed, 68.860s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 17.597s', 'individual call time for EIGEN_LDLT: CPU, 37.780s; elapsed, 37.596s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.121s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 17.597s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.780s; elapsed, 37.596s, #calls: 1', 'TOTAL : CPU, 72.730s; elapsed, 55.314s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 57.660s; elapsed, 14.674s', 'individual call time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.709s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.106s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 57.660s; elapsed, 14.674s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.106s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.709s, #calls: 1', 'TOTAL : CPU, 96.110s; elapsed, 52.489s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 121.690s; elapsed, 15.460s', 'individual call time for EIGEN_LDLT: CPU, 38.970s; elapsed, 37.714s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.093s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 121.690s; elapsed, 15.460s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.970s; elapsed, 37.714s, #calls: 1', 'TOTAL : CPU, 160.920s; elapsed, 53.267s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 275.770s; elapsed, 17.569s', 'individual call time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.708s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.100s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 275.770s; elapsed, 17.569s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.100s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.708s, #calls: 1', 'TOTAL : CPU, 316.600s; elapsed, 55.377s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.200s; elapsed, 32.237s', 'individual call time for EIGEN_LDLT: CPU, 37.670s; elapsed, 37.690s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.117s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# - nd time = 0.422451', '# - matching time = 0.0672159', '# - symmetrization time = 0.0524902', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0864451', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 31.6537', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 9.35496e+10', '# - flop rate = 2.95541 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73537e-11\trel.res = 1.86806e-15\tbw.error = 5.7415e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.142907', '# - total flops = 6.72147e+07, min = 165052, max = 6.70497e+07', '# - flop rate = 0.470338 GFlop/s', '# - bytes moved = 408.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.85588 GByte/s', '# - solve arithmetic intensity = 0.164691 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.480s; elapsed, 32.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.530s; elapsed, 32.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.480s; elapsed, 32.509s, #calls: 1', 'TOTAL : CPU, 32.480s; elapsed, 32.509s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.200s; elapsed, 32.237s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.670s; elapsed, 37.690s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.530s; elapsed, 32.521s, #calls: 1', 'TOTAL : CPU, 102.510s; elapsed, 102.565s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.840s; elapsed, 17.632s', 'individual call time for EIGEN_LDLT: CPU, 37.920s; elapsed, 37.773s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.416875', '# - matching time = 0.0671329', '# - symmetrization time = 0.0533619', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0789139', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 17.2193', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 5.43285 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.80099e-11\trel.res = 1.88391e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.125427', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.536031 GFlop/s', '# - bytes moved = 408.883 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.25993 GByte/s', '# - solve arithmetic intensity = 0.164431 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.740s; elapsed, 18.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.820s; elapsed, 18.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.740s; elapsed, 18.049s, #calls: 1', 'TOTAL : CPU, 35.740s; elapsed, 18.049s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.840s; elapsed, 17.632s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.920s; elapsed, 37.773s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.820s; elapsed, 18.061s, #calls: 1', 'TOTAL : CPU, 91.750s; elapsed, 73.577s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 60.630s; elapsed, 15.418s', 'individual call time for EIGEN_LDLT: CPU, 39.180s; elapsed, 38.663s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.431975', '# - matching time = 0.068692', '# - symmetrization time = 0.052408', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0735681', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 14.4334', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 6.48149 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79581e-11\trel.res = 1.88266e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.148777', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.451903 GFlop/s', '# - bytes moved = 408.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.74842 GByte/s', '# - solve arithmetic intensity = 0.164423 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 60.020s; elapsed, 15.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.640s; elapsed, 15.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 60.020s; elapsed, 15.299s, #calls: 1', 'TOTAL : CPU, 60.020s; elapsed, 15.299s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 60.630s; elapsed, 15.418s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.180s; elapsed, 38.663s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.640s; elapsed, 15.308s, #calls: 1', 'TOTAL : CPU, 117.670s; elapsed, 69.499s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 139.320s; elapsed, 17.783s', 'individual call time for EIGEN_LDLT: CPU, 40.220s; elapsed, 39.019s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.089s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.465776', '# - matching time = 0.073633', '# - symmetrization time = 0.0551162', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0766551', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 13.9159', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 42535, max = 9.35498e+10', '# - flop rate = 6.72251 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79316e-11\trel.res = 1.88202e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.155214', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.433163 GFlop/s', '# - bytes moved = 408.94 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.63469 GByte/s', '# - solve arithmetic intensity = 0.164408 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 116.070s; elapsed, 14.839s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.480s; elapsed, 14.851s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 116.070s; elapsed, 14.839s, #calls: 1', 'TOTAL : CPU, 116.070s; elapsed, 14.839s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 139.320s; elapsed, 17.783s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.089s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.220s; elapsed, 39.019s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.480s; elapsed, 14.851s, #calls: 1', 'TOTAL : CPU, 200.290s; elapsed, 71.742s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 296.940s; elapsed, 18.914s', 'individual call time for EIGEN_LDLT: CPU, 67.250s; elapsed, 64.608s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.102s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.51771', '# - matching time = 0.0950909', '# - symmetrization time = 0.0673761', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096982', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.609', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35508e+10, min = 43208, max = 9.35507e+10', '# - flop rate = 5.63255 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.68634e-11\trel.res = 1.85622e-15\tbw.error = 5.74638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.16012', '# - total flops = 6.72375e+07, min = 165052, max = 6.70724e+07', '# - flop rate = 0.41992 GFlop/s', '# - bytes moved = 409.091 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55491 GByte/s', '# - solve arithmetic intensity = 0.164358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 275.130s; elapsed, 17.656s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.410s; elapsed, 17.664s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 275.130s; elapsed, 17.656s, #calls: 1', 'TOTAL : CPU, 275.130s; elapsed, 17.656s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 296.940s; elapsed, 18.914s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 67.250s; elapsed, 64.608s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.410s; elapsed, 17.664s, #calls: 1', 'TOTAL : CPU, 395.070s; elapsed, 101.288s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.290s; elapsed, 34.319s', 'individual call time for EIGEN_LDLT: CPU, 39.840s; elapsed, 39.874s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.134s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.503283', '# - matching time = 0.077836', '# - symmetrization time = 0.0592399', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0661089', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.2315', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 6.14186 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73296e-11\trel.res = 1.86748e-15\tbw.error = 5.73834e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.102796', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.690147 GFlop/s', '# - bytes moved = 217.757 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11834 GByte/s', '# - solve arithmetic intensity = 0.325796 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.080s; elapsed, 16.105s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.080s; elapsed, 16.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.060s; elapsed, 16.081s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.090s; elapsed, 16.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.080s; elapsed, 16.105s, #calls: 1', 'TOTAL : CPU, 16.080s; elapsed, 16.105s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.080s; elapsed, 16.095s, #calls: 1', 'TOTAL : CPU, 16.080s; elapsed, 16.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.060s; elapsed, 16.081s, #calls: 1', 'TOTAL : CPU, 16.060s; elapsed, 16.081s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.290s; elapsed, 34.319s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.134s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.840s; elapsed, 39.874s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.090s; elapsed, 16.101s, #calls: 1', 'TOTAL : CPU, 90.350s; elapsed, 90.429s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 37.980s; elapsed, 19.221s', 'individual call time for EIGEN_LDLT: CPU, 40.370s; elapsed, 40.217s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACKCPP -2', 'CPP -2', 'CPP -2', '', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.502221', '# - matching time = 0.079648', '# - symmetrization time = 0.0596969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0704589', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.0184', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 6.22899 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71926e-11\trel.res = 1.86417e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.108056', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.656551 GFlop/s', '# - bytes moved = 217.769 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.01533 GByte/s', '# - solve arithmetic intensity = 0.325778 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.830s; elapsed, 15.910s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.820s; elapsed, 15.885s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.080s; elapsed, 15.901s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.600s; elapsed, 15.906s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.830s; elapsed, 15.910s, #calls: 1', 'TOTAL : CPU, 23.830s; elapsed, 15.910s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.820s; elapsed, 15.885s, #calls: 1', 'TOTAL : CPU, 26.820s; elapsed, 15.885s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.080s; elapsed, 15.901s, #calls: 1', 'TOTAL : CPU, 31.080s; elapsed, 15.901s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 37.980s; elapsed, 19.221s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.370s; elapsed, 40.217s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.600s; elapsed, 15.906s, #calls: 1', 'TOTAL : CPU, 95.110s; elapsed, 75.458s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 70.200s; elapsed, 17.826s', 'individual call time for EIGEN_LDLT: CPU, 75.090s; elapsed, 74.598s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.433724', '# - matching time = 0.071959', '# - symmetrization time = 0.0539112', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0734382', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 14.206', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 6.5852 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.7193e-11\trel.res = 1.86418e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.127258', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.557484 GFlop/s', '# - bytes moved = 217.784 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.71136 GByte/s', '# - solve arithmetic intensity = 0.325755 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 36.170s; elapsed, 15.038s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 44.810s; elapsed, 15.016s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 58.350s; elapsed, 15.029s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.200s; elapsed, 15.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 44.810s; elapsed, 15.016s, #calls: 1', 'TOTAL : CPU, 44.810s; elapsed, 15.016s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 36.170s; elapsed, 15.038s, #calls: 1', 'TOTAL : CPU, 36.170s; elapsed, 15.038s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 58.350s; elapsed, 15.029s, #calls: 1', 'TOTAL : CPU, 58.350s; elapsed, 15.029s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 70.200s; elapsed, 17.826s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 75.090s; elapsed, 74.598s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.200s; elapsed, 15.034s, #calls: 1', 'TOTAL : CPU, 162.710s; elapsed, 107.569s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 137.720s; elapsed, 17.551s', 'individual call time for EIGEN_LDLT: CPU, 40.400s; elapsed, 39.166s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.095s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.461751', '# - matching time = 0.083421', '# - symmetrization time = 0.0664868', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0851312', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.6386', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 42535, max = 7.04191e+10', '# - flop rate = 5.98197 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71813e-11\trel.res = 1.8639e-15\tbw.error = 5.74928e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120055', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.590932 GFlop/s', '# - bytes moved = 217.815 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.81429 GByte/s', '# - solve arithmetic intensity = 0.325709 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.030s; elapsed, 16.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 90.580s; elapsed, 16.510s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.640s; elapsed, 16.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.730s; elapsed, 16.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.640s; elapsed, 16.524s, #calls: 1', 'TOTAL : CPU, 127.640s; elapsed, 16.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 90.580s; elapsed, 16.510s, #calls: 1', 'TOTAL : CPU, 90.580s; elapsed, 16.510s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.030s; elapsed, 16.532s, #calls: 1', 'TOTAL : CPU, 59.030s; elapsed, 16.532s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 137.720s; elapsed, 17.551s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.095s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.400s; elapsed, 39.166s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.730s; elapsed, 16.530s, #calls: 1', 'TOTAL : CPU, 200.120s; elapsed, 73.341s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.810s; elapsed, 32.833s', 'individual call time for EIGEN_LDLT: CPU, 39.430s; elapsed, 39.429s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.46956', '# - matching time = 0.0716159', '# - symmetrization time = 0.0388072', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.061811', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 16.4571', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 5.77629 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01663e-10\trel.res = 2.45512e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.100274', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.809855 GFlop/s', '# - bytes moved = 211.022 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.10446 GByte/s', '# - solve arithmetic intensity = 0.384828 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.220s; elapsed, 17.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.240s; elapsed, 17.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.230s; elapsed, 17.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.230s; elapsed, 17.254s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.210s; elapsed, 17.251s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.230s; elapsed, 17.252s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.230s; elapsed, 17.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.270s; elapsed, 17.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.230s; elapsed, 17.253s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.230s; elapsed, 17.254s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.254s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.220s; elapsed, 17.258s, #calls: 1', 'TOTAL : CPU, 17.220s; elapsed, 17.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.240s; elapsed, 17.257s, #calls: 1', 'TOTAL : CPU, 17.240s; elapsed, 17.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.210s; elapsed, 17.251s, #calls: 1', 'TOTAL : CPU, 17.210s; elapsed, 17.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.230s; elapsed, 17.241s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.230s; elapsed, 17.252s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.252s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.810s; elapsed, 32.833s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.430s; elapsed, 39.429s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.270s; elapsed, 17.260s, #calls: 1', 'TOTAL : CPU, 89.630s; elapsed, 89.646s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 43.950s; elapsed, 22.275s', 'individual call time for EIGEN_LDLT: CPU, 38.280s; elapsed, 38.121s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.480459', '# - matching time = 0.081213', '# - symmetrization time = 0.0428212', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0897632', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 15.2662', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 6.22687 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01627e-10\trel.res = 2.45425e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.089186', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.910539 GFlop/s', '# - bytes moved = 211.031 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.36619 GByte/s', '# - solve arithmetic intensity = 0.384812 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.650s; elapsed, 16.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.300s; elapsed, 16.118s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.350s; elapsed, 16.113s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.310s; elapsed, 16.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.590s; elapsed, 16.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.850s; elapsed, 16.119s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.090s; elapsed, 16.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.830s; elapsed, 16.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.650s; elapsed, 16.114s, #calls: 1', 'TOTAL : CPU, 18.650s; elapsed, 16.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.350s; elapsed, 16.113s, #calls: 1', 'TOTAL : CPU, 31.350s; elapsed, 16.113s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.310s; elapsed, 16.114s, #calls: 1', 'TOTAL : CPU, 18.310s; elapsed, 16.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.300s; elapsed, 16.118s, #calls: 1', 'TOTAL : CPU, 20.300s; elapsed, 16.118s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.590s; elapsed, 16.102s, #calls: 1', 'TOTAL : CPU, 24.590s; elapsed, 16.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.090s; elapsed, 16.114s, #calls: 1', 'TOTAL : CPU, 18.090s; elapsed, 16.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.850s; elapsed, 16.119s, #calls: 1', 'TOTAL : CPU, 22.850s; elapsed, 16.119s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 43.950s; elapsed, 22.275s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.280s; elapsed, 38.121s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.830s; elapsed, 16.120s, #calls: 1', 'TOTAL : CPU, 99.230s; elapsed, 76.641s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 72.880s; elapsed, 18.513s', 'individual call time for EIGEN_LDLT: CPU, 38.520s; elapsed, 38.003s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.100s', 'CPP -2', 'CPP -2', 'CPP -2CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.533917', '# - matching time = 0.0805659', '# - symmetrization time = 0.0470691', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0791359', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 15.7734', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 6.02667 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0162e-10\trel.res = 2.45408e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.118165', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.687236 GFlop/s', '# - bytes moved = 211.042 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.78599 GByte/s', '# - solve arithmetic intensity = 0.384792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.350s; elapsed, 16.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 28.820s; elapsed, 16.710s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.590s; elapsed, 16.709s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.690s; elapsed, 16.705s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 23.520s; elapsed, 16.697sindividual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 47.520s; elapsed, 16.693s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 63.550s; elapsed, 16.705s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.930s; elapsed, 16.713s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.350s; elapsed, 16.703s, #calls: 1', 'TOTAL : CPU, 24.350s; elapsed, 16.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.690s; elapsed, 16.705s, #calls: 1', 'TOTAL : CPU, 22.690s; elapsed, 16.705s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 23.520s; elapsed, 16.697s, #calls: 1', 'TOTAL : CPU, 23.520s; elapsed, 16.697s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 47.520s; elapsed, 16.693s, #calls: 1', 'TOTAL : CPU, 47.520s; elapsed, 16.693s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.590s; elapsed, 16.709s, #calls: 1', 'TOTAL : CPU, 35.590s; elapsed, 16.709s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 28.820s; elapsed, 16.710s, #calls: 1', 'TOTAL : CPU, 28.820s; elapsed, 16.710s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 63.550s; elapsed, 16.705s, #calls: 1', 'TOTAL : CPU, 63.550s; elapsed, 16.705s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 72.880s; elapsed, 18.513s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.100s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.520s; elapsed, 38.003s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.930s; elapsed, 16.713s, #calls: 1', 'TOTAL : CPU, 130.510s; elapsed, 73.328s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.560s; elapsed, 36.612s', 'individual call time for EIGEN_LDLT: CPU, 38.770s; elapsed, 38.805s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.464769', '# - matching time = 0.071492', '# - symmetrization time = 0.020879', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0638721', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 13.6226', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 6.97817 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56982e-11\trel.res = 2.31108e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.075362', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.3417 GFlop/s', '# - bytes moved = 203.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.70082 GByte/s', '# - solve arithmetic intensity = 0.496775 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.360s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.340s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.300s; elapsed, 14.367s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.290s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.340s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.340s; elapsed, 14.367s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.310s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.320s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.340s; elapsed, 14.367s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.310s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.290s; elapsed, 14.365s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.340s; elapsed, 14.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.320s; elapsed, 14.362s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.360s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.320s; elapsed, 14.359s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.360s; elapsed, 14.372s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.320s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.320s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.340s; elapsed, 14.367s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.367s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.340s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.340s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.320s; elapsed, 14.362s, #calls: 1', 'TOTAL : CPU, 14.320s; elapsed, 14.362s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.360s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.360s; elapsed, 14.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.310s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.310s; elapsed, 14.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.360s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.360s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.300s; elapsed, 14.367s, #calls: 1', 'TOTAL : CPU, 14.300s; elapsed, 14.367s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.310s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.310s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.320s; elapsed, 14.359s, #calls: 1', 'TOTAL : CPU, 14.320s; elapsed, 14.359s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.340s; elapsed, 14.358s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.358s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.290s; elapsed, 14.365s, #calls: 1', 'TOTAL : CPU, 14.290s; elapsed, 14.365s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.340s; elapsed, 14.367s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.367s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.290s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.290s; elapsed, 14.366s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.560s; elapsed, 36.612s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.770s; elapsed, 38.805s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.360s; elapsed, 14.372s, #calls: 1', 'TOTAL : CPU, 89.810s; elapsed, 89.912s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.470s; elapsed, 19.510s', 'individual call time for EIGEN_LDLT: CPU, 39.550s; elapsed, 39.396s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.137s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.480338', '# - matching time = 0.0812621', '# - symmetrization time = 0.027029', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084131', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 14.4769', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 6.5664 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56833e-11\trel.res = 2.31072e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0731711', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.38187 GFlop/s', '# - bytes moved = 203.544 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.78175 GByte/s', '# - solve arithmetic intensity = 0.496763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.340s; elapsed, 15.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.720s; elapsed, 15.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 19.930s; elapsed, 15.281s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 22.090s; elapsed, 15.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 19.250s; elapsed, 15.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.470s; elapsed, 15.281s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.950s; elapsed, 15.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.570s; elapsed, 15.281s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.940s; elapsed, 15.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.850s; elapsed, 15.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.550s; elapsed, 15.275s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.300s; elapsed, 15.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.950s; elapsed, 15.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.840s; elapsed, 15.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 24.190s; elapsed, 15.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.960s; elapsed, 15.287s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.340s; elapsed, 15.283s, #calls: 1', 'TOTAL : CPU, 17.340s; elapsed, 15.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.470s; elapsed, 15.281s, #calls: 1', 'TOTAL : CPU, 29.470s; elapsed, 15.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.720s; elapsed, 15.282s, #calls: 1', 'TOTAL : CPU, 17.720s; elapsed, 15.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.550s; elapsed, 15.275s, #calls: 1', 'TOTAL : CPU, 17.550s; elapsed, 15.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.570s; elapsed, 15.281s, #calls: 1', 'TOTAL : CPU, 17.570s; elapsed, 15.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.300s; elapsed, 15.280s, #calls: 1', 'TOTAL : CPU, 17.300s; elapsed, 15.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.940s; elapsed, 15.279s, #calls: 1', 'TOTAL : CPU, 16.940s; elapsed, 15.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.840s; elapsed, 15.280s, #calls: 1', 'TOTAL : CPU, 16.840s; elapsed, 15.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 19.250s; elapsed, 15.282s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 19.250s; elapsed, 15.282s', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.850s; elapsed, 15.274s, #calls: 1', 'TOTAL : CPU, 17.850s; elapsed, 15.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.950s; elapsed, 15.280s, #calls: 1', 'TOTAL : CPU, 17.950s; elapsed, 15.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 19.930s; elapsed, 15.281s, #calls: 1', 'TOTAL : CPU, 19.930s; elapsed, 15.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 22.090s; elapsed, 15.282s, #calls: 1', 'TOTAL : CPU, 22.090s; elapsed, 15.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.950s; elapsed, 15.280s, #calls: 1', 'TOTAL : CPU, 16.950s; elapsed, 15.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 24.190s; elapsed, 15.276s, #calls: 1', 'TOTAL : CPU, 24.190s; elapsed, 15.276s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.470s; elapsed, 19.510s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.137s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.550s; elapsed, 39.396s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.960s; elapsed, 15.287s, #calls: 1', 'TOTAL : CPU, 94.170s; elapsed, 74.331s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 143.270s; elapsed, 143.398s', 'individual call time for EIGEN_LDLT: CPU, 292.510s; elapsed, 292.691s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.267s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 143.270s; elapsed, 143.398s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.267s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.510s; elapsed, 292.691s, #calls: 1', 'TOTAL : CPU, 436.050s; elapsed, 436.356s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 180.670s; elapsed, 91.878s', 'individual call time for EIGEN_LDLT: CPU, 292.930s; elapsed, 292.933s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.255s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 180.670s; elapsed, 91.878s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.255s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.930s; elapsed, 292.933s, #calls: 1', 'TOTAL : CPU, 473.980s; elapsed, 385.066s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 258.540s; elapsed, 65.840s', 'individual call time for EIGEN_LDLT: CPU, 293.120s; elapsed, 292.945s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.194s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 258.540s; elapsed, 65.840s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.194s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.120s; elapsed, 292.945s, #calls: 1', 'TOTAL : CPU, 552.070s; elapsed, 358.979s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 706.660s; elapsed, 90.151s', 'individual call time for EIGEN_LDLT: CPU, 294.740s; elapsed, 293.595s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.190s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 706.660s; elapsed, 90.151s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.190s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.740s; elapsed, 293.595s, #calls: 1', 'TOTAL : CPU, 1001.970s; elapsed, 383.935s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1602.230s; elapsed, 101.434s', 'individual call time for EIGEN_LDLT: CPU, 295.630s; elapsed, 293.121s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.890s; elapsed, 0.177s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1602.230s; elapsed, 101.434s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.890s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT: CPU, 295.630s; elapsed, 293.121s, #calls: 1', 'TOTAL : CPU, 1898.750s; elapsed, 394.732s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.190s; elapsed, 150.368s', 'individual call time for EIGEN_LDLT: CPU, 338.130s; elapsed, 338.313s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.405s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.12828', '# - matching time = 0.146574', '# - symmetrization time = 0.129105', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.15945', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 140', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18828e+11, min = 20466, max = 7.18828e+11', '# - flop rate = 5.13447 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03767e-10\trel.res = 2.47992e-15\tbw.error = 8.30193e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.298358', '# - total flops = 2.36136e+08, min = 120284, max = 2.36016e+08', '# - flop rate = 0.791452 GFlop/s', '# - bytes moved = 1221.21 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.09311 GByte/s', '# - solve arithmetic intensity = 0.193362 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 141.940s; elapsed, 142.027s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 142.050s; elapsed, 142.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 141.940s; elapsed, 142.027s, #calls: 1', 'TOTAL : CPU, 141.940s; elapsed, 142.027s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.190s; elapsed, 150.368s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.405s, #calls: 1', 'time for EIGEN_LDLT: CPU, 338.130s; elapsed, 338.313s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 142.050s; elapsed, 142.036s, #calls: 1', 'TOTAL : CPU, 630.770s; elapsed, 631.122s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 183.730s; elapsed, 93.434s', 'individual call time for EIGEN_LDLT: CPU, 293.460s; elapsed, 293.402s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.255s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.1405', '# - matching time = 0.12998', '# - symmetrization time = 0.133348', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.137994', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 82.9013', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18829e+11, min = 20466, max = 7.18829e+11', '# - flop rate = 8.6709 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05699e-10\trel.res = 2.52609e-15\tbw.error = 8.38845e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.370053', '# - total flops = 2.36212e+08, min = 120284, max = 2.36092e+08', '# - flop rate = 0.638321 GFlop/s', '# - bytes moved = 1223.28 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.3057 GByte/s', '# - solve arithmetic intensity = 0.193097 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 168.190s; elapsed, 84.979s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.170s; elapsed, 84.988s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 168.190s; elapsed, 84.979s, #calls: 1', 'TOTAL : CPU, 168.190s; elapsed, 84.979s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 183.730s; elapsed, 93.434s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.255s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.460s; elapsed, 293.402s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.170s; elapsed, 84.988s, #calls: 1', 'TOTAL : CPU, 563.720s; elapsed, 472.079s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 349.640s; elapsed, 89.219s', 'individual call time for EIGEN_LDLT: CPU, 347.670s; elapsed, 347.287s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.230s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.04379', '# - matching time = 0.142251', '# - symmetrization time = 0.12888', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.147299', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 87.3412', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18829e+11, min = 20466, max = 7.18829e+11', '# - flop rate = 8.23013 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02652e-10\trel.res = 2.45327e-15\tbw.error = 8.38966e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.461489', '# - total flops = 2.36212e+08, min = 120284, max = 2.36092e+08', '# - flop rate = 0.511848 GFlop/s', '# - bytes moved = 1224.53 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.65343 GByte/s', '# - solve arithmetic intensity = 0.192901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 348.600s; elapsed, 89.434s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 93.370s; elapsed, 89.445s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 348.600s; elapsed, 89.434s, #calls: 1', 'TOTAL : CPU, 348.600s; elapsed, 89.434s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 349.640s; elapsed, 89.219s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.230s, #calls: 1', 'time for EIGEN_LDLT: CPU, 347.670s; elapsed, 347.287s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 93.370s; elapsed, 89.445s, #calls: 1', 'TOTAL : CPU, 791.150s; elapsed, 526.181s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 519.820s; elapsed, 66.207s', 'individual call time for EIGEN_LDLT: CPU, 294.500s; elapsed, 293.340s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.191s', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.02498', '# - matching time = 0.14874', '# - symmetrization time = 0.147613', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.155244', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 63.955', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18829e+11, min = 21139, max = 7.18829e+11', '# - flop rate = 11.2396 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02621e-10\trel.res = 2.45254e-15\tbw.error = 8.38966e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.276627', '# - total flops = 2.36212e+08, min = 120284, max = 2.36092e+08', '# - flop rate = 0.853903 GFlop/s', '# - bytes moved = 1224.6 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.42691 GByte/s', '# - solve arithmetic intensity = 0.192889 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 512.330s; elapsed, 65.881s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.770s; elapsed, 65.888s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 512.330s; elapsed, 65.881s, #calls: 1', 'TOTAL : CPU, 512.330s; elapsed, 65.881s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 519.820s; elapsed, 66.207s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.191s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.500s; elapsed, 293.340s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.770s; elapsed, 65.888s, #calls: 1', 'TOTAL : CPU, 888.690s; elapsed, 425.626s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1014.780s; elapsed, 64.576s', 'individual call time for EIGEN_LDLT: CPU, 301.390s; elapsed, 300.326s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.940s; elapsed, 0.199s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.00899', '# - matching time = 0.220138', '# - symmetrization time = 0.163189', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.175079', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 92.4088', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18833e+11, min = 21720, max = 7.18833e+11', '# - flop rate = 7.77884 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0254e-10\trel.res = 2.45058e-15\tbw.error = 8.38966e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.405701', '# - total flops = 2.36222e+08, min = 120284, max = 2.36102e+08', '# - flop rate = 0.582256 GFlop/s', '# - bytes moved = 1224.9 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.01922 GByte/s', '# - solve arithmetic intensity = 0.19285 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1486.260s; elapsed, 94.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.070s; elapsed, 94.606s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1486.260s; elapsed, 94.590s, #calls: 1', 'TOTAL : CPU, 1486.260s; elapsed, 94.590s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1014.780s; elapsed, 64.576s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.940s; elapsed, 0.199s, #calls: 1', 'time for EIGEN_LDLT: CPU, 301.390s; elapsed, 300.326s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.070s; elapsed, 94.606s, #calls: 1', 'TOTAL : CPU, 1432.180s; elapsed, 459.707s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 147.210s; elapsed, 147.287s', 'individual call time for EIGEN_LDLT: CPU, 297.060s; elapsed, 297.137s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.281s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.11321', '# - matching time = 0.126604', '# - symmetrization time = 0.0862792', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.138134', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 97.0723', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4725e+11', '# - flop rate = 7.32555 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.12597e-11\trel.res = 2.181e-15\tbw.error = 7.73321e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.284401', '# - total flops = 3.02508e+08, min = 82374, max = 1.07552e+08', '# - flop rate = 1.06367 GFlop/s', '# - bytes moved = 501.232 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.76241 GByte/s', '# - solve arithmetic intensity = 0.603529 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 98.810s; elapsed, 98.931s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 98.810s; elapsed, 98.910s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 98.870s; elapsed, 98.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 98.920s; elapsed, 98.932s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 98.810s; elapsed, 98.931s, #calls: 1', 'TOTAL : CPU, 98.810s; elapsed, 98.931s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 98.870s; elapsed, 98.922s, #calls: 1', 'TOTAL : CPU, 98.870s; elapsed, 98.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 98.810s; elapsed, 98.910s, #calls: 1', 'TOTAL : CPU, 98.810s; elapsed, 98.910s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 147.210s; elapsed, 147.287s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.281s, #calls: 1', 'time for EIGEN_LDLT: CPU, 297.060s; elapsed, 297.137s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 98.920s; elapsed, 98.932s, #calls: 1', 'TOTAL : CPU, 543.470s; elapsed, 543.637s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 191.030s; elapsed, 97.305s', 'individual call time for EIGEN_LDLT: CPU, 307.620s; elapsed, 307.567s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.243s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.14717', '# - matching time = 0.137849', '# - symmetrization time = 0.0879951', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.123657', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 64.5627', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4725e+11', '# - flop rate = 11.0142 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.10797e-11\trel.res = 2.1767e-15\tbw.error = 7.7354e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.194512', '# - total flops = 3.02534e+08, min = 82374, max = 1.07578e+08', '# - flop rate = 1.55535 GFlop/s', '# - bytes moved = 502.373 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.58274 GByte/s', '# - solve arithmetic intensity = 0.602209 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 111.710s; elapsed, 66.364s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 82.650s; elapsed, 66.355s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 129.190s; elapsed, 66.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 67.340s; elapsed, 66.363s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 82.650s; elapsed, 66.355s, #calls: 1', 'TOTAL : CPU, 82.650s; elapsed, 66.355s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 111.710s; elapsed, 66.364s, #calls: 1', 'TOTAL : CPU, 111.710s; elapsed, 66.364s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 129.190s; elapsed, 66.341s, #calls: 1', 'TOTAL : CPU, 129.190s; elapsed, 66.341s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 191.030s; elapsed, 97.305s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.243s, #calls: 1', 'time for EIGEN_LDLT: CPU, 307.620s; elapsed, 307.567s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 67.340s; elapsed, 66.363s, #calls: 1', 'TOTAL : CPU, 566.340s; elapsed, 471.477s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 344.690s; elapsed, 88.439s', 'individual call time for EIGEN_LDLT: CPU, 316.840s; elapsed, 316.703s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.240s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.24407', '# - matching time = 0.157099', '# - symmetrization time = 0.0989389', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.123186', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 77.7695', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4725e+11', '# - flop rate = 9.14379 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.10174e-11\trel.res = 2.17521e-15\tbw.error = 7.7354e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.232172', '# - total flops = 3.02534e+08, min = 82374, max = 1.07578e+08', '# - flop rate = 1.30306 GFlop/s', '# - bytes moved = 502.387 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.16386 GByte/s', '# - solve arithmetic intensity = 0.602193 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 245.370s; elapsed, 79.736s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 122.090s; elapsed, 79.728s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 307.850s; elapsed, 79.714s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 82.720s; elapsed, 79.734s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 245.370s; elapsed, 79.736s, #calls: 1', 'TOTAL : CPU, 245.370s; elapsed, 79.736s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 122.090s; elapsed, 79.728s, #calls: 1', 'TOTAL : CPU, 122.090s; elapsed, 79.728s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 307.850s; elapsed, 79.714s, #calls: 1', 'TOTAL : CPU, 307.850s; elapsed, 79.714s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 344.690s; elapsed, 88.439s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.240s, #calls: 1', 'time for EIGEN_LDLT: CPU, 316.840s; elapsed, 316.703s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 82.720s; elapsed, 79.734s, #calls: 1', 'TOTAL : CPU, 744.740s; elapsed, 485.115s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 507.260s; elapsed, 64.805s', 'individual call time for EIGEN_LDLT: CPU, 293.870s; elapsed, 293.429s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.177s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = CPP -2', 'CPP -2', 'CPP -2', '6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.08266', '# - matching time = 0.153423', '# - symmetrization time = 0.123154', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.145828', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 89.9915', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 21139, max = 3.4725e+11', '# - flop rate = 7.90194 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.09459e-11\trel.res = 2.1735e-15\tbw.error = 7.7354e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.261675', '# - total flops = 3.02534e+08, min = 82374, max = 1.07578e+08', '# - flop rate = 1.15614 GFlop/s', '# - bytes moved = 502.416 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.92 GByte/s', '# - solve arithmetic intensity = 0.602158 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 543.590s; elapsed, 91.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 183.940s; elapsed, 91.872s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 700.390s; elapsed, 91.862s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 99.260s; elapsed, 91.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 183.940s; elapsed, 91.872s, #calls: 1', 'TOTAL : CPU, 183.940s; elapsed, 91.872s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 543.590s; elapsed, 91.879s, #calls: 1', 'TOTAL : CPU, 543.590s; elapsed, 91.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 700.390s; elapsed, 91.862s, #calls: 1', 'TOTAL : CPU, 700.390s; elapsed, 91.862s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 507.260s; elapsed, 64.805s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.870s; elapsed, 293.429s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 99.260s; elapsed, 91.880s, #calls: 1', 'TOTAL : CPU, 900.940s; elapsed, 450.291s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 149.740s; elapsed, 149.938s', 'individual call time for EIGEN_LDLT: CPU, 294.790s; elapsed, 294.983s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.269s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 8 MPI processesCPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.07697', '# - matching time = 0.123268', '# - symmetrization time = 0.0576088', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.112677', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 82.2988', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.47059e+11', '# - flop rate = 8.64056 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.20596e+11', '# --------------------------------------------', '# total = 7.20596e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.04956e-11\trel.res = 1.68477e-15\tbw.error = 5.68015e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.18976', '# - total flops = 6.03194e+08, min = 63419, max = 1.00803e+08', '# - flop rate = 3.17872 GFlop/s', '# - bytes moved = 447.163 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.35647 GByte/s', '# - solve arithmetic intensity = 1.34894 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 83.910s; elapsed, 83.965s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 83.920s; elapsed, 83.952s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 83.910s; elapsed, 83.953s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 83.720s; elapsed, 83.951s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 83.890s; elapsed, 83.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 83.890s; elapsed, 83.950s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 83.810s; elapsed, 83.946s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 83.960s; elapsed, 83.963s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 83.720s; elapsed, 83.951s, #calls: 1', 'TOTAL : CPU, 83.720s; elapsed, 83.951s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 83.910s; elapsed, 83.953s, #calls: 1', 'TOTAL : CPU, 83.910s; elapsed, 83.953s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 83.890s; elapsed, 83.955s, #calls: 1', 'TOTAL : CPU, 83.890s; elapsed, 83.955s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 83.920s; elapsed, 83.952s, #calls: 1', 'TOTAL : CPU, 83.920s; elapsed, 83.952s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 83.890s; elapsed, 83.950s, #calls: 1', 'TOTAL : CPU, 83.890s; elapsed, 83.950s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 83.810s; elapsed, 83.946s, #calls: 1', 'TOTAL : CPU, 83.810s; elapsed, 83.946s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 83.910s; elapsed, 83.965s, #calls: 1', 'TOTAL : CPU, 83.910s; elapsed, 83.965s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 149.740s; elapsed, 149.938s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.269s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.790s; elapsed, 294.983s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 83.960s; elapsed, 83.963s, #calls: 1', 'TOTAL : CPU, 528.760s; elapsed, 529.153s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 225.870s; elapsed, 114.465s', 'individual call time for EIGEN_LDLT: CPU, 327.850s; elapsed, 327.974s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.281s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.31747', '# - matching time = 0.14838', '# - symmetrization time = 0.0674', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.125913', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 60.0409', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.47059e+11', '# - flop rate = 11.8437 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.20596e+11', '# --------------------------------------------', '# total = 7.20596e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.02679e-11\trel.res = 1.67933e-15\tbw.error = 5.68236e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.154365', '# - total flops = 6.03204e+08, min = 63419, max = 1.00803e+08', '# - flop rate = 3.90765 GFlop/s', '# - bytes moved = 447.576 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.89946 GByte/s', '# - solve arithmetic intensity = 1.34771 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 88.300s; elapsed, 61.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 77.410s; elapsed, 61.943s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 70.590s; elapsed, 61.943s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 71.650s; elapsed, 61.941s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 116.860s; elapsed, 61.944s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 70.010s; elapsed, 61.946s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 108.060s; elapsed, 61.944s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 62.870s; elapsed, 61.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 108.060s; elapsed, 61.944s, #calls: 1', 'TOTAL : CPU, 108.060s; elapsed, 61.944s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 88.300s; elapsed, 61.955s, #calls: 1', 'TOTAL : CPU, 88.300s; elapsed, 61.955s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 77.410s; elapsed, 61.943s, #calls: 1', 'TOTAL : CPU, 77.410s; elapsed, 61.943s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 71.650s; elapsed, 61.941s, #calls: 1', 'TOTAL : CPU, 71.650s; elapsed, 61.941s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 116.860s; elapsed, 61.944s, #calls: 1', 'TOTAL : CPU, 116.860s; elapsed, 61.944s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 70.010s; elapsed, 61.946s, #calls: 1', 'TOTAL : CPU, 70.010s; elapsed, 61.946s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 70.590s; elapsed, 61.943s, #calls: 1', 'TOTAL : CPU, 70.590s; elapsed, 61.943s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 225.870s; elapsed, 114.465s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.281s, #calls: 1', 'time for EIGEN_LDLT: CPU, 327.850s; elapsed, 327.974s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 62.870s; elapsed, 61.959s, #calls: 1', 'TOTAL : CPU, 617.000s; elapsed, 504.679s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 280.390s; elapsed, 71.725s', 'individual call time for EIGEN_LDLT: CPU, 300.080s; elapsed, 299.656s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.207s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.11594', '# - matching time = 0.150755', '# - symmetrization time = 0.0676041', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.130359', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 62.0743', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.47059e+11', '# - flop rate = 11.4558 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.20596e+11', '# --------------------------------------------', '# total = 7.20596e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.01929e-11\trel.res = 1.67753e-15\tbw.error = 5.68236e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.172785', '# - total flops = 6.03204e+08, min = 63419, max = 1.00803e+08', '# - flop rate = 3.49107 GFlop/s', '# - bytes moved = 447.589 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.59044 GByte/s', '# - solve arithmetic intensity = 1.34767 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 138.710s; elapsed, 63.828s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 92.320s; elapsed, 63.814s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 86.740s; elapsed, 63.810s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 170.470s; elapsed, 63.809s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 236.420s; elapsed, 63.814s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 88.350s; elapsed, 63.817s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 107.390s; elapsed, 63.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 66.680s; elapsed, 63.824s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 92.320s; elapsed, 63.814s, #calls: 1', 'TOTAL : CPU, 92.320s; elapsed, 63.814s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 236.420s; elapsed, 63.814s, #calls: 1', 'TOTAL : CPU, 236.420s; elapsed, 63.814s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 138.710s; elapsed, 63.828s, #calls: 1', 'TOTAL : CPU, 138.710s; elapsed, 63.828s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 86.740s; elapsed, 63.810s, #calls: 1', 'TOTAL : CPU, 86.740s; elapsed, 63.810s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 170.470s; elapsed, 63.809s, #calls: 1', 'TOTAL : CPU, 170.470s; elapsed, 63.809s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 88.350s; elapsed, 63.817s, #calls: 1', 'TOTAL : CPU, 88.350s; elapsed, 63.817s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 107.390s; elapsed, 63.815s, #calls: 1', 'TOTAL : CPU, 107.390s; elapsed, 63.815s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 280.390s; elapsed, 71.725s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.207s, #calls: 1', 'time for EIGEN_LDLT: CPU, 300.080s; elapsed, 299.656s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 66.680s; elapsed, 63.824s, #calls: 1', 'TOTAL : CPU, 647.620s; elapsed, 435.412s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 156.040s; elapsed, 156.250s', 'individual call time for EIGEN_LDLT: CPU, 306.660s; elapsed, 306.889s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.282s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.12552', '# - matching time = 0.134906', '# - symmetrization time = 0.049021', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.12821', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 76.0076', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4671e+11', '# - flop rate = 9.35574 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.19036e+11', '# --------------------------------------------', '# total = 7.19036e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.52689e-11\trel.res = 1.55985e-15\tbw.error = 6.33399e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.126672', '# - total flops = 1.0636e+09, min = 53944, max = 1.00793e+08', '# - flop rate = 8.39646 GFlop/s', '# - bytes moved = 420.668 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.32093 GByte/s', '# - solve arithmetic intensity = 2.52835 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 77.530s; elapsed, 77.648s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 77.560s; elapsed, 77.644s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 77.590s; elapsed, 77.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 77.560s; elapsed, 77.648s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 77.550s; elapsed, 77.652s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 77.590s; elapsed, 77.652s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 77.420s; elapsed, 77.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 77.580s; elapsed, 77.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 77.620s; elapsed, 77.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 77.590s; elapsed, 77.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 77.590s; elapsed, 77.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 77.610s; elapsed, 77.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 77.570s; elapsed, 77.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 77.610s; elapsed, 77.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 77.460s; elapsed, 77.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 77.650s; elapsed, 77.653s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 77.550s; elapsed, 77.652s, #calls: 1', 'TOTAL : CPU, 77.550s; elapsed, 77.652s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 77.580s; elapsed, 77.646s, #calls: 1', 'TOTAL : CPU, 77.580s; elapsed, 77.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 77.460s; elapsed, 77.643s, #calls: 1', 'TOTAL : CPU, 77.460s; elapsed, 77.643s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 77.560s; elapsed, 77.644s, #calls: 1', 'TOTAL : CPU, 77.560s; elapsed, 77.644s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 77.530s; elapsed, 77.648s, #calls: 1', 'TOTAL : CPU, 77.530s; elapsed, 77.648s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 77.590s; elapsed, 77.646s, #calls: 1', 'TOTAL : CPU, 77.590s; elapsed, 77.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 77.560s; elapsed, 77.648s, #calls: 1', 'TOTAL : CPU, 77.560s; elapsed, 77.648s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 77.590s; elapsed, 77.643s, #calls: 1', 'TOTAL : CPU, 77.590s; elapsed, 77.643s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 77.590s; elapsed, 77.643s, #calls: 1', 'TOTAL : CPU, 77.590s; elapsed, 77.643s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 77.610s; elapsed, 77.646s, #calls: 1', 'TOTAL : CPU, 77.610s; elapsed, 77.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 77.420s; elapsed, 77.646s, #calls: 1', 'TOTAL : CPU, 77.420s; elapsed, 77.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 77.570s; elapsed, 77.643s, #calls: 1', 'TOTAL : CPU, 77.570s; elapsed, 77.643s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 77.620s; elapsed, 77.646s, #calls: 1', 'TOTAL : CPU, 77.620s; elapsed, 77.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 77.590s; elapsed, 77.652s, #calls: 1', 'TOTAL : CPU, 77.590s; elapsed, 77.652s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 77.610s; elapsed, 77.643s, #calls: 1', 'TOTAL : CPU, 77.610s; elapsed, 77.643s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 156.040s; elapsed, 156.250s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.282s, #calls: 1', 'time for EIGEN_LDLT: CPU, 306.660s; elapsed, 306.889s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 77.650s; elapsed, 77.653s, #calls: 1', 'TOTAL : CPU, 540.630s; elapsed, 541.075s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 192.160s; elapsed, 98.083s', 'individual call time for EIGEN_LDLT: CPU, 446.910s; elapsed, 447.018s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.316s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.15229', '# - matching time = 0.153208', '# - symmetrization time = 0.055331', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.144514', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 74.7298', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4671e+11', '# - flop rate = 9.51572 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.19036e+11', '# --------------------------------------------', '# total = 7.19036e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.50696e-11\trel.res = 1.55509e-15\tbw.error = 6.33399e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.124849', '# - total flops = 1.0636e+09, min = 53944, max = 1.00793e+08', '# - flop rate = 8.51907 GFlop/s', '# - bytes moved = 420.681 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.36952 GByte/s', '# - solve arithmetic intensity = 2.52827 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 84.050s; elapsed, 76.439s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 83.800s; elapsed, 76.437s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 95.410s; elapsed, 76.445s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 83.110s; elapsed, 76.444s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 121.160s; elapsed, 76.437s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 144.190s; elapsed, 76.442s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 83.780s; elapsed, 76.440s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 90.720s; elapsed, 76.442s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 83.480s; elapsed, 76.440s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 82.510s; elapsed, 76.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 82.410s; elapsed, 76.442s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 83.400s; elapsed, 76.451s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 81.940s; elapsed, 76.457s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 95.290s; elapsed, 76.460s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 81.750s; elapsed, 76.449s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 77.330s; elapsed, 76.460s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 83.800s; elapsed, 76.437s, #calls: 1', 'TOTAL : CPU, 83.800s; elapsed, 76.437s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 83.110s; elapsed, 76.444s, #calls: 1', 'TOTAL : CPU, 83.110s; elapsed, 76.444s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 83.400s; elapsed, 76.451s, #calls: 1', 'TOTAL : CPU, 83.400s; elapsed, 76.451s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 95.290s; elapsed, 76.460s, #calls: 1', 'TOTAL : CPU, 95.290s; elapsed, 76.460s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 84.050s; elapsed, 76.439s, #calls: 1', 'TOTAL : CPU, 84.050s; elapsed, 76.439s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 83.480s; elapsed, 76.440s, #calls: 1', 'TOTAL : CPU, 83.480s; elapsed, 76.440s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 83.780s; elapsed, 76.440s, #calls: 1', 'TOTAL : CPU, 83.780s; elapsed, 76.440s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 144.190s; elapsed, 76.442s, #calls: 1', 'TOTAL : CPU, 144.190s; elapsed, 76.442s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 95.410s; elapsed, 76.445s, #calls: 1', 'TOTAL : CPU, 95.410s; elapsed, 76.445s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 90.720s; elapsed, 76.442s, #calls: 1', 'TOTAL : CPU, 90.720s; elapsed, 76.442s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 121.160s; elapsed, 76.437s, #calls: 1', 'TOTAL : CPU, 121.160s; elapsed, 76.437s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 82.410s; elapsed, 76.442s, #calls: 1', 'TOTAL : CPU, 82.410s; elapsed, 76.442s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 82.510s; elapsed, 76.441s, #calls: 1', 'TOTAL : CPU, 82.510s; elapsed, 76.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 81.940s; elapsed, 76.457s, #calls: 1', 'TOTAL : CPU, 81.940s; elapsed, 76.457s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 81.750s; elapsed, 76.449s, #calls: 1', 'TOTAL : CPU, 81.750s; elapsed, 76.449s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 192.160s; elapsed, 98.083s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.316s, #calls: 1', 'time for EIGEN_LDLT: CPU, 446.910s; elapsed, 447.018s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 77.330s; elapsed, 76.460s, #calls: 1', 'TOTAL : CPU, 716.860s; elapsed, 621.877s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 141.610s; elapsed, 141.725s', 'individual call time for EIGEN_LDLT: CPU, 292.480s; elapsed, 292.681s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.278s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 141.610s; elapsed, 141.725s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.278s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.480s; elapsed, 292.681s, #calls: 1', 'TOTAL : CPU, 434.360s; elapsed, 434.684s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 158.640s; elapsed, 79.647s', 'individual call time for EIGEN_LDLT: CPU, 293.900s; elapsed, 293.938s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.247s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 158.640s; elapsed, 79.647s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.247s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.900s; elapsed, 293.938s, #calls: 1', 'TOTAL : CPU, 452.920s; elapsed, 373.832s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 280.410s; elapsed, 70.887s', 'individual call time for EIGEN_LDLT: CPU, 293.610s; elapsed, 293.255s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.254s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 280.410s; elapsed, 70.887s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.254s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.610s; elapsed, 293.255s, #calls: 1', 'TOTAL : CPU, 574.540s; elapsed, 364.395s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 473.450s; elapsed, 60.373s', 'individual call time for EIGEN_LDLT: CPU, 293.190s; elapsed, 292.802s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.170s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 473.450s; elapsed, 60.373s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.170s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.190s; elapsed, 292.802s, #calls: 1', 'TOTAL : CPU, 767.150s; elapsed, 353.345s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 952.520s; elapsed, 60.297s', 'individual call time for EIGEN_LDLT: CPU, 305.190s; elapsed, 302.772s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.940s; elapsed, 0.185s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 952.520s; elapsed, 60.297s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.940s; elapsed, 0.185s, #calls: 1', 'time for EIGEN_LDLT: CPU, 305.190s; elapsed, 302.772s, #calls: 1', 'TOTAL : CPU, 1258.650s; elapsed, 363.254s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 149.000s; elapsed, 149.135s', 'individual call time for EIGEN_LDLT: CPU, 389.180s; elapsed, 389.333s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.420s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.56776', '# - matching time = 0.131552', '# - symmetrization time = 0.113881', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.182462', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 85.7369', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.39212 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12487e-10\trel.res = 2.68831e-15\tbw.error = 6.87881e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.207274', '# - total flops = 2.36268e+08, min = 4.69568e+07, max = 1.89311e+08', '# - flop rate = 1.13988 GFlop/s', '# - bytes moved = 525.018 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.53297 GByte/s', '# - solve arithmetic intensity = 0.450019 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 86.890s; elapsed, 86.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.860s; elapsed, 86.984s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 86.890s; elapsed, 86.966s, #calls: 1', 'TOTAL : CPU, 86.890s; elapsed, 86.966s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 149.000s; elapsed, 149.135s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.420s, #calls: 1', 'time for EIGEN_LDLT: CPU, 389.180s; elapsed, 389.333s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.860s; elapsed, 86.984s, #calls: 1', 'TOTAL : CPU, 625.460s; elapsed, 625.872s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 179.700s; elapsed, 91.342s', 'individual call time for EIGEN_LDLT: CPU, 369.840s; elapsed, 369.874s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.307s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.609775', '# - matching time = 0.129897', '# - symmetrization time = 0.116029', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.193751', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 83.9515', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.57059 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12473e-10\trel.res = 2.68797e-15\tbw.error = 6.87847e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.197684', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 1.19543 GFlop/s', '# - bytes moved = 526.314 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.6624 GByte/s', '# - solve arithmetic intensity = 0.449003 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 136.860s; elapsed, 85.222s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 168.020s; elapsed, 85.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 136.860s; elapsed, 85.222s, #calls: 1', 'TOTAL : CPU, 136.860s; elapsed, 85.222s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 179.700s; elapsed, 91.342s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.307s, #calls: 1', 'time for EIGEN_LDLT: CPU, 369.840s; elapsed, 369.874s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 168.020s; elapsed, 85.241s, #calls: 1', 'TOTAL : CPU, 718.010s; elapsed, 546.763s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 370.230s; elapsed, 93.568s', 'individual call time for EIGEN_LDLT: CPU, 304.250s; elapsed, 303.935s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.236s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.602328', '# - matching time = 0.144056', '# - symmetrization time = 0.121829', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.208224', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 62.3783', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 11.5347 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12402e-10\trel.res = 2.68628e-15\tbw.error = 6.87847e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.246262', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.959614 GFlop/s', '# - bytes moved = 526.329 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.13727 GByte/s', '# - solve arithmetic intensity = 0.44899 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 210.870s; elapsed, 63.727s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 244.070s; elapsed, 63.750s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 210.870s; elapsed, 63.727s, #calls: 1', 'TOTAL : CPU, 210.870s; elapsed, 63.727s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 370.230s; elapsed, 93.568s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.236s, #calls: 1', 'time for EIGEN_LDLT: CPU, 304.250s; elapsed, 303.935s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 244.070s; elapsed, 63.750s, #calls: 1', 'TOTAL : CPU, 919.030s; elapsed, 461.489s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 611.910s; elapsed, 78.062s', 'individual call time for EIGEN_LDLT: CPU, 307.390s; elapsed, 306.662s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.200s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.594699', '# - matching time = 0.152848', '# - symmetrization time = 0.138616', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.23336', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 79.5824', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 9.04113 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12404e-10\trel.res = 2.68632e-15\tbw.error = 6.87887e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.259088', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.912109 GFlop/s', '# - bytes moved = 526.362 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.03159 GByte/s', '# - solve arithmetic intensity = 0.448962 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 439.520s; elapsed, 80.997s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 625.090s; elapsed, 81.019s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 439.520s; elapsed, 80.997s, #calls: 1', 'TOTAL : CPU, 439.520s; elapsed, 80.997s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 611.910s; elapsed, 78.062s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.200s, #calls: 1', 'time for EIGEN_LDLT: CPU, 307.390s; elapsed, 306.662s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 625.090s; elapsed, 81.019s, #calls: 1', 'TOTAL : CPU, 1545.020s; elapsed, 465.943s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1292.460s; elapsed, 81.874s', 'individual call time for EIGEN_LDLT: CPU, 297.310s; elapsed, 294.859s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.980s; elapsed, 0.189s', 'CPP -2', '# Initializing STRUMPACKCPP -2', '', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.618963', '# - matching time = 0.170063', '# - symmetrization time = 0.137581', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.243485', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 105.031', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 6.85047 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12389e-10\trel.res = 2.68597e-15\tbw.error = 6.87887e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.418363', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.56486 GFlop/s', '# - bytes moved = 526.428 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.2583 GByte/s', '# - solve arithmetic intensity = 0.448906 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1193.420s; elapsed, 106.694s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1641.840s; elapsed, 106.712s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1193.420s; elapsed, 106.694s, #calls: 1', 'TOTAL : CPU, 1193.420s; elapsed, 106.694s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1292.460s; elapsed, 81.874s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.980s; elapsed, 0.189s, #calls: 1', 'time for EIGEN_LDLT: CPU, 297.310s; elapsed, 294.859s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1641.840s; elapsed, 106.712s, #calls: 1', 'TOTAL : CPU, 3232.590s; elapsed, 483.634s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 157.630s; elapsed, 157.742s', 'individual call time for EIGEN_LDLT: CPU, 313.280s; elapsed, 313.539s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.394s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.582414', '# - matching time = 0.135348', '# - symmetrization time = 0.0798221', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.175818', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 84.5521', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16766e+08, max = 5.51962e+11', '# - flop rate = 8.45701 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.75852e-11\trel.res = 2.09319e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.367124', '# - total flops = 2.50759e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 0.683037 GFlop/s', '# - bytes moved = 456.661 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.24389 GByte/s', '# - solve arithmetic intensity = 0.549114 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.910s; elapsed, 85.945s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 85.890s; elapsed, 85.943s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 85.760s; elapsed, 85.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.890s; elapsed, 85.955s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.910s; elapsed, 85.945s, #calls: 1', 'TOTAL : CPU, 85.910s; elapsed, 85.945s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 85.890s; elapsed, 85.943s, #calls: 1', 'TOTAL : CPU, 85.890s; elapsed, 85.943s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 85.760s; elapsed, 85.928s, #calls: 1', 'TOTAL : CPU, 85.760s; elapsed, 85.928s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 157.630s; elapsed, 157.742s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.394s, #calls: 1', 'time for EIGEN_LDLT: CPU, 313.280s; elapsed, 313.539s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.890s; elapsed, 85.955s, #calls: 1', 'TOTAL : CPU, 557.200s; elapsed, 557.629s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 285.470s; elapsed, 144.124s', 'individual call time for EIGEN_LDLT: CPU, 828.930s; elapsed, 829.305s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.342s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.602572', '# - matching time = 0.185682', '# - symmetrization time = 0.0742941', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.181473', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 66.6783', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16766e+08, max = 5.51962e+11', '# - flop rate = 10.724 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40934e-11\trel.res = 2.00974e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.345228', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 0.726391 GFlop/s', '# - bytes moved = 457.114 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.32409 GByte/s', '# - solve arithmetic intensity = 0.548595 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 80.550s; elapsed, 68.121s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 119.630s; elapsed, 68.115s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 109.710s; elapsed, 68.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 131.800s; elapsed, 68.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 80.550s; elapsed, 68.121s, #calls: 1', 'TOTAL : CPU, 80.550s; elapsed, 68.121s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 109.710s; elapsed, 68.120s, #calls: 1', 'TOTAL : CPU, 109.710s; elapsed, 68.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 119.630s; elapsed, 68.115s, #calls: 1', 'TOTAL : CPU, 119.630s; elapsed, 68.115s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 285.470s; elapsed, 144.124s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.342s, #calls: 1', 'time for EIGEN_LDLT: CPU, 828.930s; elapsed, 829.305s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 131.800s; elapsed, 68.138s, #calls: 1', 'TOTAL : CPU, 1246.680s; elapsed, 1041.910s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 436.010s; elapsed, 113.368s', 'individual call time for EIGEN_LDLT: CPU, 726.280s; elapsed, 726.824s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.270s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.639624', '# - matching time = 0.144666', '# - symmetrization time = 0.080354', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.207279', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 81.1593', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16767e+08, max = 5.51962e+11', '# - flop rate = 8.81054 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40844e-11\trel.res = 2.00952e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.299015', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 0.838655 GFlop/s', '# - bytes moved = 457.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.52877 GByte/s', '# - solve arithmetic intensity = 0.54858 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 259.020s; elapsed, 82.586s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 193.500s; elapsed, 82.579s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 113.490s; elapsed, 82.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 306.280s; elapsed, 82.598s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 259.020s; elapsed, 82.586s, #calls: 1', 'TOTAL : CPU, 259.020s; elapsed, 82.586s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 193.500s; elapsed, 82.579s, #calls: 1', 'TOTAL : CPU, 193.500s; elapsed, 82.579s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 113.490s; elapsed, 82.589s, #calls: 1', 'TOTAL : CPU, 113.490s; elapsed, 82.589s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 436.010s; elapsed, 113.368s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.270s, #calls: 1', 'time for EIGEN_LDLT: CPU, 726.280s; elapsed, 726.824s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 306.280s; elapsed, 82.598s, #calls: 1', 'TOTAL : CPU, 1469.110s; elapsed, 923.059s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 826.270s; elapsed, 105.239s', 'individual call time for EIGEN_LDLT: CPU, 296.370s; elapsed, 296.040s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.660s; elapsed, 0.215s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4CPP -2', 'CPP -2', 'CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.662766', '# - matching time = 0.153435', '# - symmetrization time = 0.095221', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.204315', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 116.873', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16767e+08, max = 5.51962e+11', '# - flop rate = 6.11822 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40817e-11\trel.res = 2.00946e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.257613', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 0.973438 GFlop/s', '# - bytes moved = 457.151 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.77456 GByte/s', '# - solve arithmetic intensity = 0.54855 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 489.580s; elapsed, 118.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 369.630s; elapsed, 118.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 213.360s; elapsed, 118.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 903.850s; elapsed, 118.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 489.580s; elapsed, 118.276s, #calls: 1', 'TOTAL : CPU, 489.580s; elapsed, 118.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 369.630s; elapsed, 118.268s, #calls: 1', 'TOTAL : CPU, 369.630s; elapsed, 118.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 213.360s; elapsed, 118.279s, #calls: 1', 'TOTAL : CPU, 213.360s; elapsed, 118.279s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 826.270s; elapsed, 105.239s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.660s; elapsed, 0.215s, #calls: 1', 'time for EIGEN_LDLT: CPU, 296.370s; elapsed, 296.040s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 903.850s; elapsed, 118.285s, #calls: 1', 'TOTAL : CPU, 2027.150s; elapsed, 519.779s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 205.990s; elapsed, 206.510s', 'individual call time for EIGEN_LDLT: CPU, 428.490s; elapsed, 428.833s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.294s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.566017', '# - matching time = 0.128769', '# - symmetrization time = 0.0554199', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.164468', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 136.187', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84773e+07, max = 5.53144e+11', '# - flop rate = 5.27222 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.52645e-11\trel.res = 1.79874e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.210446', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 1.36087 GFlop/s', '# - bytes moved = 432.794 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05656 GByte/s', '# - solve arithmetic intensity = 0.661723 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 137.280s; elapsed, 137.347s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 137.290s; elapsed, 137.347s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 137.250s; elapsed, 137.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 137.230s; elapsed, 137.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 137.250s; elapsed, 137.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 137.150s; elapsed, 137.349s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 137.270s; elapsed, 137.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 136.970s; elapsed, 137.356s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 137.280s; elapsed, 137.347s, #calls: 1', 'TOTAL : CPU, 137.280s; elapsed, 137.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 137.290s; elapsed, 137.347s, #calls: 1', 'TOTAL : CPU, 137.290s; elapsed, 137.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 137.250s; elapsed, 137.339s, #calls: 1', 'TOTAL : CPU, 137.250s; elapsed, 137.339s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 137.270s; elapsed, 137.344s, #calls: 1', 'TOTAL : CPU, 137.270s; elapsed, 137.344s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 137.150s; elapsed, 137.349s, #calls: 1', 'TOTAL : CPU, 137.150s; elapsed, 137.349s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 137.230s; elapsed, 137.339s, #calls: 1', 'TOTAL : CPU, 137.230s; elapsed, 137.339s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 137.250s; elapsed, 137.339s, #calls: 1', 'TOTAL : CPU, 137.250s; elapsed, 137.339s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 205.990s; elapsed, 206.510s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.294s, #calls: 1', 'time for EIGEN_LDLT: CPU, 428.490s; elapsed, 428.833s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 136.970s; elapsed, 137.356s, #calls: 1', 'TOTAL : CPU, 771.740s; elapsed, 772.993s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 353.800s; elapsed, 182.038s', 'individual call time for EIGEN_LDLT: CPU, 551.610s; elapsed, 552.335s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.314s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# - nd time = 0.619913', '# - matching time = 0.147258', '# - symmetrization time = 0.0594411', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.186284', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 164.384', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84773e+07, max = 5.53144e+11', '# - flop rate = 4.36787 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49551e-11\trel.res = 1.79134e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.286097', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 1.00102 GFlop/s', '# - bytes moved = 432.807 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.5128 GByte/s', '# - solve arithmetic intensity = 0.661703 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 188.590s; elapsed, 165.773s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 192.150s; elapsed, 165.764s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 188.730s; elapsed, 165.771s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 188.080s; elapsed, 165.770s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 234.000s; elapsed, 165.773s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 233.870s; elapsed, 165.766s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 213.180s; elapsed, 165.765s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 317.590s; elapsed, 165.779s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 233.870s; elapsed, 165.766s, #calls: 1', 'TOTAL : CPU, 233.870s; elapsed, 165.766s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 188.730s; elapsed, 165.771s, #calls: 1', 'TOTAL : CPU, 188.730s; elapsed, 165.771s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 192.150s; elapsed, 165.764s, #calls: 1', 'TOTAL : CPU, 192.150s; elapsed, 165.764s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 188.590s; elapsed, 165.773s, #calls: 1', 'TOTAL : CPU, 188.590s; elapsed, 165.773s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 234.000s; elapsed, 165.773s, #calls: 1', 'TOTAL : CPU, 234.000s; elapsed, 165.773s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 188.080s; elapsed, 165.770s, #calls: 1', 'TOTAL : CPU, 188.080s; elapsed, 165.770s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 213.180s; elapsed, 165.765s, #calls: 1', 'TOTAL : CPU, 213.180s; elapsed, 165.765s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 353.800s; elapsed, 182.038s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.314s, #calls: 1', 'time for EIGEN_LDLT: CPU, 551.610s; elapsed, 552.335s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 317.590s; elapsed, 165.779s, #calls: 1', 'TOTAL : CPU, 1223.460s; elapsed, 900.467s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 831.180s; elapsed, 214.470s', 'individual call time for EIGEN_LDLT: CPU, 702.500s; elapsed, 703.317s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.305s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.641791', '# - matching time = 0.165633', '# - symmetrization time = 0.064028', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.207586', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 168.698', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84791e+07, max = 5.53144e+11', '# - flop rate = 4.25618 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49343e-11\trel.res = 1.79085e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.252246', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 1.13536 GFlop/s', '# - bytes moved = 432.824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.71588 GByte/s', '# - solve arithmetic intensity = 0.661677 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 190.670s; elapsed, 170.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 198.750s; elapsed, 170.059s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 247.940s; elapsed, 170.059s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 191.880s; elapsed, 170.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 437.860s; elapsed, 170.065s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 317.440s; elapsed, 170.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 193.360s; elapsed, 170.064s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 645.680s; elapsed, 170.072s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 247.940s; elapsed, 170.059s, #calls: 1', 'TOTAL : CPU, 247.940s; elapsed, 170.059s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 437.860s; elapsed, 170.065s, #calls: 1', 'TOTAL : CPU, 437.860s; elapsed, 170.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 193.360s; elapsed, 170.064s, #calls: 1', 'TOTAL : CPU, 193.360s; elapsed, 170.064s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 190.670s; elapsed, 170.062s, #calls: 1', 'TOTAL : CPU, 190.670s; elapsed, 170.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 198.750s; elapsed, 170.059s, #calls: 1', 'TOTAL : CPU, 198.750s; elapsed, 170.059s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 317.440s; elapsed, 170.057s, #calls: 1', 'TOTAL : CPU, 317.440s; elapsed, 170.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 191.880s; elapsed, 170.061s, #calls: 1', 'TOTAL : CPU, 191.880s; elapsed, 170.061s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 831.180s; elapsed, 214.470s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.305s, #calls: 1', 'time for EIGEN_LDLT: CPU, 702.500s; elapsed, 703.317s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 645.680s; elapsed, 170.072s, #calls: 1', 'TOTAL : CPU, 2179.980s; elapsed, 1088.164s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 160.580s; elapsed, 160.735s', 'individual call time for EIGEN_LDLT: CPU, 313.310s; elapsed, 313.547s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.321s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,149', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.682611', '# - matching time = 0.147931', '# - symmetrization time = 0.052649', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,149', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.186143', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 902.044 MB', '# - factor time = 126.539', '# - factor nonzeros = 112,755,472', '# - factor memory = 902.044 MB', '# - total flops = 7.1691e+11, min = 9.76849e+07, max = 5.51315e+11', '# - flop rate = 5.66554 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2397e+11', '# --------------------------------------------', '# total = 7.2397e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.99952e-11\trel.res = 1.91179e-15\tbw.error = 5.68886e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.114264', '# - total flops = 3.67095e+08, min = 217887, max = 1.87487e+08', '# - flop rate = 3.21269 GFlop/s', '# - bytes moved = 405.893 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.55224 GByte/s', '# - solve arithmetic intensity = 0.904413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.610s; elapsed, 127.741s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 127.670s; elapsed, 127.741s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 127.660s; elapsed, 127.741s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 127.650s; elapsed, 127.742s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 127.620s; elapsed, 127.740s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 127.640s; elapsed, 127.741s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 127.620s; elapsed, 127.740s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 127.570s; elapsed, 127.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 119.510s; elapsed, 127.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 127.660s; elapsed, 127.740s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 127.690s; elapsed, 127.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 120.840s; elapsed, 127.738s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 127.620s; elapsed, 127.736s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 118.950s; elapsed, 127.750s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 118.110s; elapsed, 127.746s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 119.010s; elapsed, 127.757s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 127.650s; elapsed, 127.742s, #calls: 1', 'TOTAL : CPU, 127.650s; elapsed, 127.742s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 127.640s; elapsed, 127.741s, #calls: 1', 'TOTAL : CPU, 127.640s; elapsed, 127.741s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 127.620s; elapsed, 127.740s, #calls: 1', 'TOTAL : CPU, 127.620s; elapsed, 127.740s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 119.510s; elapsed, 127.737s, #calls: 1', 'TOTAL : CPU, 119.510s; elapsed, 127.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 127.690s; elapsed, 127.737s, #calls: 1', 'TOTAL : CPU, 127.690s; elapsed, 127.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 127.570s; elapsed, 127.737s, #calls: 1', 'TOTAL : CPU, 127.570s; elapsed, 127.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 127.670s; elapsed, 127.741s, #calls: 1', 'TOTAL : CPU, 127.670s; elapsed, 127.741s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.610s; elapsed, 127.741s, #calls: 1', 'TOTAL : CPU, 127.610s; elapsed, 127.741s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 127.660s; elapsed, 127.741s, #calls: 1', 'TOTAL : CPU, 127.660s; elapsed, 127.741s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 127.620s; elapsed, 127.740s, #calls: 1', 'TOTAL : CPU, 127.620s; elapsed, 127.740s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 120.840s; elapsed, 127.738s, #calls: 1', 'TOTAL : CPU, 120.840s; elapsed, 127.738s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 127.660s; elapsed, 127.740s, #calls: 1', 'TOTAL : CPU, 127.660s; elapsed, 127.740s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 127.620s; elapsed, 127.736s, #calls: 1', 'TOTAL : CPU, 127.620s; elapsed, 127.736s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 118.110s; elapsed, 127.746s, #calls: 1', 'TOTAL : CPU, 118.110s; elapsed, 127.746s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 118.950s; elapsed, 127.750s, #calls: 1', 'TOTAL : CPU, 118.950s; elapsed, 127.750s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 160.580s; elapsed, 160.735s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.321s, #calls: 1', 'time for EIGEN_LDLT: CPU, 313.310s; elapsed, 313.547s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 119.010s; elapsed, 127.757s, #calls: 1', 'TOTAL : CPU, 593.230s; elapsed, 602.360s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 307.130s; elapsed, 159.107s', 'individual call time for EIGEN_LDLT: CPU, 344.790s; elapsed, 344.887s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.350s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4CPP -2', 'CPP -2', ' = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,149', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.74583', '# - matching time = 0.181826', '# - symmetrization time = 0.0588651', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,149', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.204101', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 902.044 MB', '# - factor time = 114.223', '# - factor nonzeros = 112,755,472', '# - factor memory = 902.044 MB', '# - total flops = 7.1691e+11, min = 9.76849e+07, max = 5.51315e+11', '# - flop rate = 6.27642 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2397e+11', '# --------------------------------------------', '# total = 7.2397e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.98842e-11\trel.res = 1.90914e-15\tbw.error = 5.67767e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.148492', '# - total flops = 3.67095e+08, min = 217887, max = 1.87487e+08', '# - flop rate = 2.47215 GFlop/s', '# - bytes moved = 405.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.7335 GByte/s', '# - solve arithmetic intensity = 0.904391 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 123.010s; elapsed, 115.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 124.750s; elapsed, 115.592sindividual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 147.610s; elapsed, 115.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 122.400s; elapsed, 115.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 122.420s; elapsed, 115.591s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 121.530s; elapsed, 115.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 123.580s; elapsed, 115.587s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 140.540s; elapsed, 115.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 123.270s; elapsed, 115.592s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 123.320s; elapsed, 115.591s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 121.560s; elapsed, 115.591s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 161.440s; elapsed, 115.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 123.550s; elapsed, 115.591s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 166.620s; elapsed, 115.593s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 125.190s; elapsed, 115.589s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 218.930s; elapsed, 115.599s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 122.400s; elapsed, 115.590s, #calls: 1', 'TOTAL : CPU, 122.400s; elapsed, 115.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 124.750s; elapsed, 115.592s, #calls: 1', 'TOTAL : CPU, 124.750s; elapsed, 115.592s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 121.530s; elapsed, 115.589s, #calls: 1', 'TOTAL : CPU, 121.530s; elapsed, 115.589s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 123.010s; elapsed, 115.589s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 147.610s; elapsed, 115.588s, #calls: 1', 'TOTAL : CPU, 147.610s; elapsed, 115.588s', 'Exiting profiler', 'TOTAL : CPU, 123.010s; elapsed, 115.589s', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 123.320s; elapsed, 115.591s, #calls: 1', 'TOTAL : CPU, 123.320s; elapsed, 115.591s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 123.580s; elapsed, 115.587s, #calls: 1', 'TOTAL : CPU, 123.580s; elapsed, 115.587s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 121.560s; elapsed, 115.591s, #calls: 1', 'TOTAL : CPU, 121.560s; elapsed, 115.591s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 122.420s; elapsed, 115.591s, #calls: 1', 'TOTAL : CPU, 122.420s; elapsed, 115.591s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 166.620s; elapsed, 115.593s, #calls: 1', 'TOTAL : CPU, 166.620s; elapsed, 115.593s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 125.190s; elapsed, 115.589s, #calls: 1', 'TOTAL : CPU, 125.190s; elapsed, 115.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 161.440s; elapsed, 115.589s, #calls: 1', 'TOTAL : CPU, 161.440s; elapsed, 115.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 123.270s; elapsed, 115.592s, #calls: 1', 'TOTAL : CPU, 123.270s; elapsed, 115.592s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 140.540s; elapsed, 115.589s, #calls: 1', 'TOTAL : CPU, 140.540s; elapsed, 115.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 123.550s; elapsed, 115.591s, #calls: 1', 'TOTAL : CPU, 123.550s; elapsed, 115.591s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 307.130s; elapsed, 159.107s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.350s, #calls: 1', 'time for EIGEN_LDLT: CPU, 344.790s; elapsed, 344.887s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 218.930s; elapsed, 115.599s, #calls: 1', 'TOTAL : CPU, 871.380s; elapsed, 619.943s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 42.780s; elapsed, 42.798s', 'individual call time for EIGEN_LDLT: CPU, 38.050s; elapsed, 38.073s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.118s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 42.780s; elapsed, 42.798s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.050s; elapsed, 38.073s, #calls: 1', 'TOTAL : CPU, 80.950s; elapsed, 80.989s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.510s; elapsed, 28.480s', 'individual call time for EIGEN_LDLT: CPU, 37.750s; elapsed, 37.588s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.108s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.510s; elapsed, 28.480s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.108s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.750s; elapsed, 37.588s, #calls: 1', 'TOTAL : CPU, 93.410s; elapsed, 66.176s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 114.240s; elapsed, 29.577s', 'individual call time for EIGEN_LDLT: CPU, 38.160s; elapsed, 37.651s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.105s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 114.240s; elapsed, 29.577s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.160s; elapsed, 37.651s, #calls: 1', 'TOTAL : CPU, 152.610s; elapsed, 67.333s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 245.250s; elapsed, 31.102s', 'individual call time for EIGEN_LDLT: CPU, 42.840s; elapsed, 41.763s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.109s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 245.250s; elapsed, 31.102s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.840s; elapsed, 41.763s, #calls: 1', 'TOTAL : CPU, 288.350s; elapsed, 72.974s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 424.590s; elapsed, 27.088s', 'individual call time for EIGEN_LDLT: CPU, 40.120s; elapsed, 37.850s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.090s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 424.590s; elapsed, 27.088s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.090s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.120s; elapsed, 37.850s, #calls: 1', 'TOTAL : CPU, 465.100s; elapsed, 65.028s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 42.840s; elapsed, 42.886s', 'individual call time for EIGEN_LDLT: CPU, 38.170s; elapsed, 38.203s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.118s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.242678', '# - matching time = 0.066668', '# - symmetrization time = 0.0467761', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0900922', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 29.8927', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 3.25991 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.93647e-11\trel.res = 2.39962e-15\tbw.error = 4.56252e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.125751', '# - total flops = 6.69055e+07, min = 2.31678e+07, max = 4.37377e+07', '# - flop rate = 0.532047 GFlop/s', '# - bytes moved = 286.981 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.28214 GByte/s', '# - solve arithmetic intensity = 0.233135 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.460s; elapsed, 30.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.500s; elapsed, 30.517s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.460s; elapsed, 30.502s, #calls: 1', 'TOTAL : CPU, 30.460s; elapsed, 30.502s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 42.840s; elapsed, 42.886s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.170s; elapsed, 38.203s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.500s; elapsed, 30.517s, #calls: 1', 'TOTAL : CPU, 111.630s; elapsed, 111.725s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 53.820s; elapsed, 27.547s', 'individual call time for EIGEN_LDLT: CPU, 37.820s; elapsed, 37.664s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.114s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.248161', '# - matching time = 0.0649562', '# - symmetrization time = 0.045748', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.092654', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 22.1984', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 4.38985 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.91496e-11\trel.res = 2.39443e-15\tbw.error = 4.6452e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.111765', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.598689 GFlop/s', '# - bytes moved = 287.3 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57058 GByte/s', '# - solve arithmetic intensity = 0.232901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 44.170s; elapsed, 22.810s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 42.170s; elapsed, 22.826s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 44.170s; elapsed, 22.810s, #calls: 1', 'TOTAL : CPU, 44.170s; elapsed, 22.810s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 53.820s; elapsed, 27.547s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.114s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.820s; elapsed, 37.664s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 42.170s; elapsed, 22.826s, #calls: 1', 'TOTAL : CPU, 133.980s; elapsed, 88.151s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 113.920s; elapsed, 29.557s', 'individual call time for EIGEN_LDLT: CPU, 39.610s; elapsed, 39.161s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.101s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.261128', '# - matching time = 0.0695879', '# - symmetrization time = 0.049494', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096493', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 27.6983', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 3.51819 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.90918e-11\trel.res = 2.39303e-15\tbw.error = 4.6452e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120251', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.55644 GFlop/s', '# - bytes moved = 287.316 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.38931 GByte/s', '# - solve arithmetic intensity = 0.232888 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 80.840s; elapsed, 28.340s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.850s; elapsed, 28.354s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 80.840s; elapsed, 28.340s, #calls: 1', 'TOTAL : CPU, 80.840s; elapsed, 28.340s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 113.920s; elapsed, 29.557s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.610s; elapsed, 39.161s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.850s; elapsed, 28.354s, #calls: 1', 'TOTAL : CPU, 265.560s; elapsed, 97.173s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 261.160s; elapsed, 33.551s', 'individual call time for EIGEN_LDLT: CPU, 43.000s; elapsed, 41.943s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 8CPP -2', ' OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.254063', '# - matching time = 0.077409', '# - symmetrization time = 0.0521879', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.11219', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 27.1422', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 3.59026 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9097e-11\trel.res = 2.39316e-15\tbw.error = 4.64314e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.128021', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.522668 GFlop/s', '# - bytes moved = 287.348 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.24454 GByte/s', '# - solve arithmetic intensity = 0.232862 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 155.140s; elapsed, 27.821s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 218.300s; elapsed, 27.840s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 155.140s; elapsed, 27.821s, #calls: 1', 'TOTAL : CPU, 155.140s; elapsed, 27.821s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 261.160s; elapsed, 33.551s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 43.000s; elapsed, 41.943s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 218.300s; elapsed, 27.840s, #calls: 1', 'TOTAL : CPU, 522.730s; elapsed, 103.439s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 521.030s; elapsed, 33.185s', 'individual call time for EIGEN_LDLT: CPU, 41.310s; elapsed, 39.011s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.102s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.293341', '# - matching time = 0.0914221', '# - symmetrization time = 0.058394', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.123988', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 32.9068', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 2.96132 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.90639e-11\trel.res = 2.39236e-15\tbw.error = 4.64314e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.147921', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.452352 GFlop/s', '# - bytes moved = 287.413 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94302 GByte/s', '# - solve arithmetic intensity = 0.232809 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 419.950s; elapsed, 33.691s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 522.970s; elapsed, 33.705s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 419.950s; elapsed, 33.691s, #calls: 1', 'TOTAL : CPU, 419.950s; elapsed, 33.691s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 521.030s; elapsed, 33.185s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 41.310s; elapsed, 39.011s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 522.970s; elapsed, 33.705s, #calls: 1', 'TOTAL : CPU, 1085.940s; elapsed, 106.003s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 67.950s; elapsed, 68.169s', 'individual call time for EIGEN_LDLT: CPU, 43.850s; elapsed, 43.880s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.146s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.267001', '# - matching time = 0.072989', '# - symmetrization time = 0.0366101', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084537', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 28.0968', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73096e+06, max = 5.90189e+10', '# - flop rate = 3.4277 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49226e-11\trel.res = 1.80935e-15\tbw.error = 4.2417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0769029', '# - total flops = 7.3905e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.961018 GFlop/s', '# - bytes moved = 252.69 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.28583 GByte/s', '# - solve arithmetic intensity = 0.292473 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 28.570s; elapsed, 28.661s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.640s; elapsed, 28.658s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 28.600s; elapsed, 28.648s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.530s; elapsed, 28.666s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 28.570s; elapsed, 28.661s, #calls: 1', 'TOTAL : CPU, 28.570s; elapsed, 28.661s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.640s; elapsed, 28.658s, #calls: 1', 'TOTAL : CPU, 28.640s; elapsed, 28.658s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 28.600s; elapsed, 28.648s, #calls: 1', 'TOTAL : CPU, 28.600s; elapsed, 28.648s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 67.950s; elapsed, 68.169s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.146s, #calls: 1', 'time for EIGEN_LDLT: CPU, 43.850s; elapsed, 43.880s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.530s; elapsed, 28.666s, #calls: 1', 'TOTAL : CPU, 140.480s; elapsed, 140.861s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 87.800s; elapsed, 44.560s', 'individual call time for EIGEN_LDLT: CPU, 42.170s; elapsed, 42.037s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.283188', '# - matching time = 0.0787721', '# - symmetrization time = 0.0385282', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0943911', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 29.3333', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73096e+06, max = 5.90189e+10', '# - flop rate = 3.28321 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47973e-11\trel.res = 1.80633e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.088002', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.839915 GFlop/s', '# - bytes moved = 252.882 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.87359 GByte/s', '# - solve arithmetic intensity = 0.292287 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 41.230s; elapsed, 29.950s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.020s; elapsed, 29.946s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 44.190s; elapsed, 29.937s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 58.910s; elapsed, 29.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 41.230s; elapsed, 29.950s, #calls: 1', 'TOTAL : CPU, 41.230s; elapsed, 29.950s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.020s; elapsed, 29.946s, #calls: 1', 'TOTAL : CPU, 32.020s; elapsed, 29.946s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 44.190s; elapsed, 29.937s, #calls: 1', 'TOTAL : CPU, 44.190s; elapsed, 29.937s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 87.800s; elapsed, 44.560s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.170s; elapsed, 42.037s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 58.910s; elapsed, 29.954s, #calls: 1', 'TOTAL : CPU, 189.050s; elapsed, 116.672s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 112.350s; elapsed, 29.106s', 'individual call time for EIGEN_LDLT: CPU, 43.090s; elapsed, 42.590s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.108s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.283755', '# - matching time = 0.0808182', '# - symmetrization time = 0.037225', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.098608', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 27.9096', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73175e+06, max = 5.90189e+10', '# - flop rate = 3.45069 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47955e-11\trel.res = 1.80628e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0996161', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.74199 GFlop/s', '# - bytes moved = 252.89 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.53865 GByte/s', '# - solve arithmetic intensity = 0.292277 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 63.920s; elapsed, 28.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 88.170s; elapsed, 28.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.340s; elapsed, 28.542s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.850s; elapsed, 28.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 88.170s; elapsed, 28.531s, #calls: 1', 'TOTAL : CPU, 88.170s; elapsed, 28.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.340s; elapsed, 28.542s, #calls: 1', 'TOTAL : CPU, 34.340s; elapsed, 28.542s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 63.920s; elapsed, 28.544s, #calls: 1', 'TOTAL : CPU, 63.920s; elapsed, 28.544s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 112.350s; elapsed, 29.106s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.108s, #calls: 1', 'time for EIGEN_LDLT: CPU, 43.090s; elapsed, 42.590s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.850s; elapsed, 28.548s, #calls: 1', 'TOTAL : CPU, 267.500s; elapsed, 100.351s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 239.970s; elapsed, 31.046s', 'individual call time for EIGEN_LDLT: CPU, 44.740s; elapsed, 43.681s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.271542', '# - matching time = 0.081696', '# - symmetrization time = 0.0566239', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.11455', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 31.0597', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73175e+06, max = 5.90189e+10', '# - flop rate = 3.10071 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47952e-11\trel.res = 1.80628e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.105457', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.700893 GFlop/s', '# - bytes moved = 252.909 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.39821 GByte/s', '# - solve arithmetic intensity = 0.292256 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 99.210s; elapsed, 31.729s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 193.560s; elapsed, 31.714s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 44.380s; elapsed, 31.724s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 246.170s; elapsed, 31.726s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 193.560s; elapsed, 31.714s, #calls: 1', 'TOTAL : CPU, 193.560s; elapsed, 31.714s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 99.210s; elapsed, 31.729s, #calls: 1', 'TOTAL : CPU, 99.210s; elapsed, 31.729s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 44.380s; elapsed, 31.724s, #calls: 1', 'TOTAL : CPU, 44.380s; elapsed, 31.724s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 239.970s; elapsed, 31.046s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 44.740s; elapsed, 43.681s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 246.170s; elapsed, 31.726s, #calls: 1', 'TOTAL : CPU, 531.210s; elapsed, 106.569s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 56.200s; elapsed, 56.348s', 'individual call time for EIGEN_LDLT: CPU, 40.330s; elapsed, 40.354s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.127s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.289625', '# - matching time = 0.070632', '# - symmetrization time = 0.0345049', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0808921', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 28.2778', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33333e+06, max = 5.89486e+10', '# - flop rate = 3.45237 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.77384e-11\trel.res = 1.63586e-15\tbw.error = 4.40647e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.084012', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 1.09371 GFlop/s', '# - bytes moved = 243.937 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.9036 GByte/s', '# - solve arithmetic intensity = 0.376674 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 28.800s; elapsed, 28.857s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 28.810s; elapsed, 28.862s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.820s; elapsed, 28.857s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 28.840s; elapsed, 28.858s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 28.850s; elapsed, 28.859s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 28.830s; elapsed, 28.858s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 28.810s; elapsed, 28.849s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.780s; elapsed, 28.865s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 28.810s; elapsed, 28.849s, #calls: 1', 'TOTAL : CPU, 28.810s; elapsed, 28.849s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 28.850s; elapsed, 28.859s, #calls: 1', 'TOTAL : CPU, 28.850s; elapsed, 28.859s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 28.800s; elapsed, 28.857s, #calls: 1', 'TOTAL : CPU, 28.800s; elapsed, 28.857s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.820s; elapsed, 28.857s, #calls: 1', 'TOTAL : CPU, 28.820s; elapsed, 28.857s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 28.840s; elapsed, 28.858s, #calls: 1', 'TOTAL : CPU, 28.840s; elapsed, 28.858s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 28.810s; elapsed, 28.862s, #calls: 1', 'TOTAL : CPU, 28.810s; elapsed, 28.862s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 28.830s; elapsed, 28.858s, #calls: 1', 'TOTAL : CPU, 28.830s; elapsed, 28.858s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 56.200s; elapsed, 56.348s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.127s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.330s; elapsed, 40.354s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.780s; elapsed, 28.865s, #calls: 1', 'TOTAL : CPU, 125.440s; elapsed, 125.694s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 72.030s; elapsed, 36.972s', 'individual call time for EIGEN_LDLT: CPU, 41.960s; elapsed, 41.818s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.351707', '# - matching time = 0.0752411', '# - symmetrization time = 0.0344861', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0973518', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 27.385', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33333e+06, max = 5.89486e+10', '# - flop rate = 3.56491 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.76835e-11\trel.res = 1.63453e-15\tbw.error = 4.41108e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0820172', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 1.12031 GFlop/s', '# - bytes moved = 243.947 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.97434 GByte/s', '# - solve arithmetic intensity = 0.376659 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 35.390s; elapsed, 28.060s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 30.280s; elapsed, 28.047s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 47.300s; elapsed, 28.056s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 39.690s; elapsed, 28.054s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.060s; elapsed, 28.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 29.610s; elapsed, 28.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 29.830s; elapsed, 28.056s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 55.020s; elapsed, 28.063s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 47.300s; elapsed, 28.056s, #calls: 1', 'TOTAL : CPU, 47.300s; elapsed, 28.056s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 30.280s; elapsed, 28.047s, #calls: 1', 'TOTAL : CPU, 30.280s; elapsed, 28.047s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 29.610s; elapsed, 28.057s, #calls: 1', 'TOTAL : CPU, 29.610s; elapsed, 28.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.060s; elapsed, 28.057s, #calls: 1', 'TOTAL : CPU, 30.060s; elapsed, 28.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 35.390s; elapsed, 28.060s, #calls: 1', 'TOTAL : CPU, 35.390s; elapsed, 28.060s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 29.830s; elapsed, 28.056s, #calls: 1', 'TOTAL : CPU, 29.830s; elapsed, 28.056s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 39.690s; elapsed, 28.054s, #calls: 1', 'TOTAL : CPU, 39.690s; elapsed, 28.054s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 72.030s; elapsed, 36.972s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 41.960s; elapsed, 41.818s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 55.020s; elapsed, 28.063s, #calls: 1', 'TOTAL : CPU, 169.170s; elapsed, 106.975s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 115.510s; elapsed, 30.025s', 'individual call time for EIGEN_LDLT: CPU, 39.930s; elapsed, 39.447s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.104s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.30026', '# - matching time = 0.0813029', '# - symmetrization time = 0.0379519', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.105072', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 29.9836', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33405e+06, max = 5.89486e+10', '# - flop rate = 3.25596 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.76723e-11\trel.res = 1.63426e-15\tbw.error = 4.40185e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0921812', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 0.996784 GFlop/s', '# - bytes moved = 243.959 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.64652 GByte/s', '# - solve arithmetic intensity = 0.37664 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 67.640s; elapsed, 30.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 51.810s; elapsed, 30.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 35.680s; elapsed, 30.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 36.380s; elapsed, 30.637s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 37.780s; elapsed, 30.625s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 87.920s; elapsed, 30.636s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.290s; elapsed, 30.636s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 119.420s; elapsed, 30.642s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 35.680s; elapsed, 30.635s, #calls: 1', 'TOTAL : CPU, 35.680s; elapsed, 30.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 36.380s; elapsed, 30.637s, #calls: 1', 'TOTAL : CPU, 36.380s; elapsed, 30.637s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 67.640s; elapsed, 30.634s, #calls: 1', 'TOTAL : CPU, 67.640s; elapsed, 30.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 51.810s; elapsed, 30.640s, #calls: 1', 'TOTAL : CPU, 51.810s; elapsed, 30.640s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 87.920s; elapsed, 30.636s, #calls: 1', 'TOTAL : CPU, 87.920s; elapsed, 30.636s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 37.780s; elapsed, 30.625s, #calls: 1', 'TOTAL : CPU, 37.780s; elapsed, 30.625s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.290s; elapsed, 30.636s, #calls: 1', 'TOTAL : CPU, 37.290s; elapsed, 30.636s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 115.510s; elapsed, 30.025s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.930s; elapsed, 39.447s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 119.420s; elapsed, 30.642s, #calls: 1', 'TOTAL : CPU, 275.050s; elapsed, 100.217s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 59.140s; elapsed, 59.256s', 'individual call time for EIGEN_LDLT: CPU, 40.890s; elapsed, 40.921s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.138s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.292161', '# - matching time = 0.0782099', '# - symmetrization time = 0.0194199', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.077369', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 23.5319', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.58592e+06, max = 5.89302e+10', '# - flop rate = 4.14108 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01444e+11', '# --------------------------------------------', '# total = 1.01444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.46798e-11\trel.res = 2.28648e-15\tbw.error = 3.3904e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.092304', '# - total flops = 1.34224e+08, min = 40130, max = 4.34304e+07', '# - flop rate = 1.45415 GFlop/s', '# - bytes moved = 235.766 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55424 GByte/s', '# - solve arithmetic intensity = 0.569308 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 24.050s; elapsed, 24.125s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 24.020s; elapsed, 24.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 24.060s; elapsed, 24.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 24.090s; elapsed, 24.121s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 24.060s; elapsed, 24.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 24.090s; elapsed, 24.116s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 24.080s; elapsed, 24.115s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.060s; elapsed, 24.121s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 24.030s; elapsed, 24.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 24.030s; elapsed, 24.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 24.070s; elapsed, 24.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.110s; elapsed, 24.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 24.090s; elapsed, 24.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.080s; elapsed, 24.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 24.070s; elapsed, 24.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 24.070s; elapsed, 24.128s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 24.050s; elapsed, 24.125s, #calls: 1', 'TOTAL : CPU, 24.050s; elapsed, 24.125s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 24.070s; elapsed, 24.120s, #calls: 1', 'TOTAL : CPU, 24.070s; elapsed, 24.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.110s; elapsed, 24.122s, #calls: 1', 'TOTAL : CPU, 24.110s; elapsed, 24.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 24.030s; elapsed, 24.123s, #calls: 1', 'TOTAL : CPU, 24.030s; elapsed, 24.123s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 24.060s; elapsed, 24.120s, #calls: 1', 'TOTAL : CPU, 24.060s; elapsed, 24.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 24.090s; elapsed, 24.122s, #calls: 1', 'TOTAL : CPU, 24.090s; elapsed, 24.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 24.060s; elapsed, 24.123s, #calls: 1', 'TOTAL : CPU, 24.060s; elapsed, 24.123s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.080s; elapsed, 24.122s, #calls: 1', 'TOTAL : CPU, 24.080s; elapsed, 24.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 24.090s; elapsed, 24.116s, #calls: 1', 'TOTAL : CPU, 24.090s; elapsed, 24.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 24.030s; elapsed, 24.117s, #calls: 1', 'TOTAL : CPU, 24.030s; elapsed, 24.117s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 24.090s; elapsed, 24.121s, #calls: 1', 'TOTAL : CPU, 24.090s; elapsed, 24.121s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 24.070s; elapsed, 24.122s, #calls: 1', 'TOTAL : CPU, 24.070s; elapsed, 24.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 24.020s; elapsed, 24.123s, #calls: 1', 'TOTAL : CPU, 24.020s; elapsed, 24.123s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 24.080s; elapsed, 24.115s, #calls: 1', 'TOTAL : CPU, 24.080s; elapsed, 24.115s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.060s; elapsed, 24.121s, #calls: 1', 'TOTAL : CPU, 24.060s; elapsed, 24.121s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 59.140s; elapsed, 59.256s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.138s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.890s; elapsed, 40.921s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 24.070s; elapsed, 24.128s, #calls: 1', 'TOTAL : CPU, 124.240s; elapsed, 124.444s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 83.820s; elapsed, 42.500s', 'individual call time for EIGEN_LDLT: CPU, 42.880s; elapsed, 42.743s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.301943', '# - matching time = 0.0791581', '# - symmetrization time = 0.019819', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.091558', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 32.2385', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.58592e+06, max = 5.89302e+10', '# - flop rate = 3.02271 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01444e+11', '# --------------------------------------------', '# total = 1.01444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.46625e-11\trel.res = 2.28606e-15\tbw.error = 3.3904e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0695741', '# - total flops = 1.34224e+08, min = 40130, max = 4.34304e+07', '# - flop rate = 1.92922 GFlop/s', '# - bytes moved = 235.775 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.38883 GByte/s', '# - solve arithmetic intensity = 0.569288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 35.290s; elapsed, 32.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 52.130s; elapsed, 32.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 40.090s; elapsed, 32.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.300s; elapsed, 32.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 35.240s; elapsed, 32.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 38.940s; elapsed, 32.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 34.370s; elapsed, 32.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 34.220s; elapsed, 32.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 35.530s; elapsed, 32.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 43.470s; elapsed, 32.827s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 35.000s; elapsed, 32.829s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 34.680s; elapsed, 32.827s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 34.800s; elapsed, 32.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 34.350s; elapsed, 32.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 35.040s; elapsed, 32.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.270s; elapsed, 32.839s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 35.530s; elapsed, 32.834s, #calls: 1', 'TOTAL : CPU, 35.530s; elapsed, 32.834s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 35.000s; elapsed, 32.829s, #calls: 1', 'TOTAL : CPU, 35.000s; elapsed, 32.829s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 52.130s; elapsed, 32.835s, #calls: 1', 'TOTAL : CPU, 52.130s; elapsed, 32.835s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.300s; elapsed, 32.833s, #calls: 1', 'TOTAL : CPU, 35.300s; elapsed, 32.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 34.800s; elapsed, 32.833s, #calls: 1', 'TOTAL : CPU, 34.800s; elapsed, 32.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 43.470s; elapsed, 32.827s, #calls: 1', 'TOTAL : CPU, 43.470s; elapsed, 32.827s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 35.240s; elapsed, 32.832s, #calls: 1', 'TOTAL : CPU, 35.240s; elapsed, 32.832s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 35.290s; elapsed, 32.834s, #calls: 1', 'TOTAL : CPU, 35.290s; elapsed, 32.834s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 34.350s; elapsed, 32.833s, #calls: 1', 'TOTAL : CPU, 34.350s; elapsed, 32.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 35.040s; elapsed, 32.833s, #calls: 1', 'TOTAL : CPU, 35.040s; elapsed, 32.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 34.220s; elapsed, 32.833s, #calls: 1', 'TOTAL : CPU, 34.220s; elapsed, 32.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 40.090s; elapsed, 32.835s, #calls: 1', 'TOTAL : CPU, 40.090s; elapsed, 32.835s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 34.370s; elapsed, 32.833s, #calls: 1', 'TOTAL : CPU, 34.370s; elapsed, 32.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 34.680s; elapsed, 32.827s, #calls: 1', 'TOTAL : CPU, 34.680s; elapsed, 32.827s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 38.940s; elapsed, 32.835s, #calls: 1', 'TOTAL : CPU, 38.940s; elapsed, 32.835s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 83.820s; elapsed, 42.500s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.880s; elapsed, 42.743s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.270s; elapsed, 32.839s, #calls: 1', 'TOTAL : CPU, 191.130s; elapsed, 118.202s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.010s; elapsed, 146.160s', 'individual call time for EIGEN_LDLT: CPU, 292.530s; elapsed, 292.710s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.010s; elapsed, 146.160s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.274s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.530s; elapsed, 292.710s, #calls: 1', 'TOTAL : CPU, 438.820s; elapsed, 439.144s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 263.870s; elapsed, 133.730s', 'individual call time for EIGEN_LDLT: CPU, 292.700s; elapsed, 292.723s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.261s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 263.870s; elapsed, 133.730s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.261s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.700s; elapsed, 292.723s, #calls: 1', 'TOTAL : CPU, 556.980s; elapsed, 426.714s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 486.750s; elapsed, 123.187s', 'individual call time for EIGEN_LDLT: CPU, 322.230s; elapsed, 322.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.262s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 486.750s; elapsed, 123.187s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.262s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.230s; elapsed, 322.230s, #calls: 1', 'TOTAL : CPU, 809.530s; elapsed, 445.679s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 934.880s; elapsed, 120.238s', 'individual call time for EIGEN_LDLT: CPU, 293.650s; elapsed, 293.248s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.257s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 934.880s; elapsed, 120.238s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.257s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.650s; elapsed, 293.248s, #calls: 1', 'TOTAL : CPU, 1229.240s; elapsed, 413.743s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1230.240s; elapsed, 79.070s', 'individual call time for EIGEN_LDLT: CPU, 313.740s; elapsed, 311.203s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.170s; elapsed, 0.219s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1230.240s; elapsed, 79.070s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.170s; elapsed, 0.219s, #calls: 1', 'time for EIGEN_LDLT: CPU, 313.740s; elapsed, 311.203s, #calls: 1', 'TOTAL : CPU, 1545.150s; elapsed, 390.493s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.120s; elapsed, 146.257s', 'individual call time for EIGEN_LDLT: CPU, 292.250s; elapsed, 292.466s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.274s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.19617', '# - matching time = 0.131757', '# - symmetrization time = 0.081625', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.158017', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 203.019', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17167e+11, min = 222721, max = 7.17167e+11', '# - flop rate = 3.53251 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15237e-10\trel.res = 2.75403e-15\tbw.error = 7.73582e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.324624', '# - total flops = 2.35974e+08, min = 302329, max = 2.35671e+08', '# - flop rate = 0.726914 GFlop/s', '# - bytes moved = 1221.18 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.76182 GByte/s', '# - solve arithmetic intensity = 0.193235 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 204.700s; elapsed, 205.173s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 205.170s; elapsed, 205.153s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 204.700s; elapsed, 205.173s, #calls: 1', 'TOTAL : CPU, 204.700s; elapsed, 205.173s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.120s; elapsed, 146.257s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.274s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.250s; elapsed, 292.466s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 205.170s; elapsed, 205.153s, #calls: 1', 'TOTAL : CPU, 643.820s; elapsed, 644.149s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 261.530s; elapsed, 132.449s', 'individual call time for EIGEN_LDLT: CPU, 420.850s; elapsed, 420.916s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.318s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.22835', '# - matching time = 0.134171', '# - symmetrization time = 0.0836952', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.167373', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 92.6548', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17168e+11, min = 222721, max = 7.17168e+11', '# - flop rate = 7.74021 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.19346e-10\trel.res = 2.85224e-15\tbw.error = 7.68484e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.240223', '# - total flops = 2.36048e+08, min = 302329, max = 2.35746e+08', '# - flop rate = 0.98262 GFlop/s', '# - bytes moved = 1223.21 MB, min = 0 MB, max = 0 MB', '# - byte rate = 5.09198 GByte/s', '# - solve arithmetic intensity = 0.192974 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 186.100s; elapsed, 94.709s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 95.770s; elapsed, 94.685s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 186.100s; elapsed, 94.709s, #calls: 1', 'TOTAL : CPU, 186.100s; elapsed, 94.709s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 261.530s; elapsed, 132.449s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.318s, #calls: 1', 'time for EIGEN_LDLT: CPU, 420.850s; elapsed, 420.916s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 95.770s; elapsed, 94.685s, #calls: 1', 'TOTAL : CPU, 778.620s; elapsed, 648.368s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 469.040s; elapsed, 119.001s', 'individual call time for EIGEN_LDLT: CPU, 322.800s; elapsed, 322.513s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.262s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.2872', '# - matching time = 0.152478', '# - symmetrization time = 0.0868559', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.171576', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 125.033', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17168e+11, min = 222721, max = 7.17168e+11', '# - flop rate = 5.73584 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18787e-10\trel.res = 2.83887e-15\tbw.error = 7.68484e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.528305', '# - total flops = 2.36048e+08, min = 302329, max = 2.35746e+08', '# - flop rate = 0.446802 GFlop/s', '# - bytes moved = 1224.43 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.31765 GByte/s', '# - solve arithmetic intensity = 0.192782 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 497.260s; elapsed, 127.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 131.770s; elapsed, 127.546s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 497.260s; elapsed, 127.569s, #calls: 1', 'TOTAL : CPU, 497.260s; elapsed, 127.569s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 469.040s; elapsed, 119.001s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.262s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.800s; elapsed, 322.513s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 131.770s; elapsed, 127.546s, #calls: 1', 'TOTAL : CPU, 924.130s; elapsed, 569.323s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 907.040s; elapsed, 117.116s', 'individual call time for EIGEN_LDLT: CPU, 649.870s; elapsed, 649.929s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.840s; elapsed, 0.266s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.24802', '# - matching time = 0.168573', '# - symmetrization time = 0.10358', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.232407', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 108.58', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17168e+11, min = 224411, max = 7.17168e+11', '# - flop rate = 6.60496 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18709e-10\trel.res = 2.837e-15\tbw.error = 7.69021e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.426475', '# - total flops = 2.36048e+08, min = 302329, max = 2.35746e+08', '# - flop rate = 0.553486 GFlop/s', '# - bytes moved = 1224.5 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.87121 GByte/s', '# - solve arithmetic intensity = 0.192771 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 861.980s; elapsed, 111.107s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 121.740s; elapsed, 111.080s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 861.980s; elapsed, 111.107s, #calls: 1', 'TOTAL : CPU, 861.980s; elapsed, 111.107s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 907.040s; elapsed, 117.116s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.840s; elapsed, 0.266s, #calls: 1', 'time for EIGEN_LDLT: CPU, 649.870s; elapsed, 649.929s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 121.740s; elapsed, 111.080s, #calls: 1', 'TOTAL : CPU, 1679.490s; elapsed, 878.391s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1924.850s; elapsed, 122.694s', 'individual call time for EIGEN_LDLT: CPU, 631.900s; elapsed, 630.983s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.170s; elapsed, 0.223s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', '16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.2464', '# - matching time = 0.171759', '# - symmetrization time = 0.1026', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.194932', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 131.019', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17172e+11, min = 226606, max = 7.17172e+11', '# - flop rate = 5.47382 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18656e-10\trel.res = 2.83575e-15\tbw.error = 7.69557e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.442774', '# - total flops = 2.36057e+08, min = 302329, max = 2.35755e+08', '# - flop rate = 0.533133 GFlop/s', '# - bytes moved = 1224.79 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.76618 GByte/s', '# - solve arithmetic intensity = 0.192732 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2089.430s; elapsed, 133.519s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 155.380s; elapsed, 133.486s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2089.430s; elapsed, 133.519s, #calls: 1', 'TOTAL : CPU, 2089.430s; elapsed, 133.519s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1924.850s; elapsed, 122.694s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.170s; elapsed, 0.223s, #calls: 1', 'time for EIGEN_LDLT: CPU, 631.900s; elapsed, 630.983s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 155.380s; elapsed, 133.486s, #calls: 1', 'TOTAL : CPU, 2713.300s; elapsed, 887.386s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 200.500s; elapsed, 200.936s', 'individual call time for EIGEN_LDLT: CPU, 329.830s; elapsed, 330.025s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.334s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.2249', '# - matching time = 0.140628', '# - symmetrization time = 0.100987', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.111343', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 91.8362', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 223014, max = 5.53072e+11', '# - flop rate = 7.85008 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.60424e-11\trel.res = 2.29531e-15\tbw.error = 7.82955e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198408', '# - total flops = 2.51898e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 1.2696 GFlop/s', '# - bytes moved = 462.411 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33061 GByte/s', '# - solve arithmetic intensity = 0.544749 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 93.590s; elapsed, 93.732s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 93.560s; elapsed, 93.710s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 93.500s; elapsed, 93.699s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 93.670s; elapsed, 93.715s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 93.560s; elapsed, 93.710s, #calls: 1', 'TOTAL : CPU, 93.560s; elapsed, 93.710s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 93.500s; elapsed, 93.699s, #calls: 1', 'TOTAL : CPU, 93.500s; elapsed, 93.699s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 93.590s; elapsed, 93.732s, #calls: 1', 'TOTAL : CPU, 93.590s; elapsed, 93.732s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 200.500s; elapsed, 200.936s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.334s, #calls: 1', 'time for EIGEN_LDLT: CPU, 329.830s; elapsed, 330.025s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 93.670s; elapsed, 93.715s, #calls: 1', 'TOTAL : CPU, 624.330s; elapsed, 625.011s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 253.580s; elapsed, 128.463s', 'individual call time for EIGEN_LDLT: CPU, 331.930s; elapsed, 331.972s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.287s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.19131', '# - matching time = 0.137186', '# - symmetrization time = 0.099227', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.119332', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 123.248', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 223014, max = 5.53072e+11', '# - flop rate = 5.84935 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.66947e-11\trel.res = 2.31089e-15\tbw.error = 7.81571e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.26263', '# - total flops = 2.51909e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 0.959179 GFlop/s', '# - bytes moved = 462.869 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.76244 GByte/s', '# - solve arithmetic intensity = 0.544234 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 165.880s; elapsed, 125.209s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 172.580s; elapsed, 125.188s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 244.440s; elapsed, 125.178s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 126.220s; elapsed, 125.193s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 172.580s; elapsed, 125.188s, #calls: 1', 'TOTAL : CPU, 172.580s; elapsed, 125.188s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 165.880s; elapsed, 125.209s, #calls: 1', 'TOTAL : CPU, 165.880s; elapsed, 125.209s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 244.440s; elapsed, 125.178s, #calls: 1', 'TOTAL : CPU, 244.440s; elapsed, 125.178s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 253.580s; elapsed, 128.463s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.287s, #calls: 1', 'time for EIGEN_LDLT: CPU, 331.930s; elapsed, 331.972s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 126.220s; elapsed, 125.193s, #calls: 1', 'TOTAL : CPU, 712.160s; elapsed, 585.915s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 554.880s; elapsed, 141.307s', 'individual call time for EIGEN_LDLT: CPU, 375.000s; elapsed, 374.919s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.248s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.23689', '# - matching time = 0.142184', '# - symmetrization time = 0.10253', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.148521', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 105.852', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 223014, max = 5.53072e+11', '# - flop rate = 6.81062 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.58014e-11\trel.res = 2.28955e-15\tbw.error = 7.811e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.322105', '# - total flops = 2.51909e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 0.782072 GFlop/s', '# - bytes moved = 462.883 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.43706 GByte/s', '# - solve arithmetic intensity = 0.544218 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 242.560s; elapsed, 107.980s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 235.390s; elapsed, 107.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 417.610s; elapsed, 107.948s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.430s; elapsed, 107.963s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 235.390s; elapsed, 107.958s, #calls: 1', 'TOTAL : CPU, 235.390s; elapsed, 107.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 242.560s; elapsed, 107.980s, #calls: 1', 'TOTAL : CPU, 242.560s; elapsed, 107.980s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 417.610s; elapsed, 107.948s, #calls: 1', 'TOTAL : CPU, 417.610s; elapsed, 107.948s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 554.880s; elapsed, 141.307s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.248s, #calls: 1', 'time for EIGEN_LDLT: CPU, 375.000s; elapsed, 374.919s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.430s; elapsed, 107.963s, #calls: 1', 'TOTAL : CPU, 1041.850s; elapsed, 624.436s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 912.560s; elapsed, 116.790s', 'individual call time for EIGEN_LDLT: CPU, 304.170s; elapsed, 304.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.233s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.26206', '# - matching time = 0.165944', '# - symmetrization time = 0.115711', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.159095', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 107.918', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 224964, max = 5.53072e+11', '# - flop rate = 6.68027 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.57902e-11\trel.res = 2.28928e-15\tbw.error = 7.811e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.349773', '# - total flops = 2.51909e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 0.720208 GFlop/s', '# - bytes moved = 462.912 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.32346 GByte/s', '# - solve arithmetic intensity = 0.544184 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 484.880s; elapsed, 110.149s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 846.830s; elapsed, 110.116s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 608.450s; elapsed, 110.125s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.620s; elapsed, 110.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 608.450s; elapsed, 110.125s, #calls: 1', 'TOTAL : CPU, 608.450s; elapsed, 110.125s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 846.830s; elapsed, 110.116s, #calls: 1', 'TOTAL : CPU, 846.830s; elapsed, 110.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 484.880s; elapsed, 110.149s, #calls: 1', 'TOTAL : CPU, 484.880s; elapsed, 110.149s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 912.560s; elapsed, 116.790s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.233s, #calls: 1', 'time for EIGEN_LDLT: CPU, 304.170s; elapsed, 304.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.620s; elapsed, 110.129s, #calls: 1', 'TOTAL : CPU, 1336.060s; elapsed, 531.202s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 5040.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 214.860s; elapsed, 215.411s', 'individual call time for EIGEN_LDLT: CPU, 324.990s; elapsed, 325.138s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.333s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.31568', '# - matching time = 0.140383', '# - symmetrization time = 0.0706091', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.116775', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 74.9153', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.53864e+11', '# - flop rate = 9.58249 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.26672e+11', '# --------------------------------------------', '# total = 7.26672e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.05517e-11\trel.res = 1.9251e-15\tbw.error = 6.12855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.161408', '# - total flops = 2.86208e+08, min = 226725, max = 1.89071e+08', '# - flop rate = 1.7732 GFlop/s', '# - bytes moved = 434.975 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.69488 GByte/s', '# - solve arithmetic intensity = 0.657987 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 76.740s; elapsed, 76.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 76.740s; elapsed, 76.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 76.760s; elapsed, 76.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 76.560s; elapsed, 76.811s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 76.720s; elapsed, 76.818s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 76.790s; elapsed, 76.820s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 76.750s; elapsed, 76.810s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.820s; elapsed, 76.829s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 76.750s; elapsed, 76.810s, #calls: 1', 'TOTAL : CPU, 76.750s; elapsed, 76.810s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 76.740s; elapsed, 76.833s, #calls: 1', 'TOTAL : CPU, 76.740s; elapsed, 76.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 76.740s; elapsed, 76.833s, #calls: 1', 'TOTAL : CPU, 76.740s; elapsed, 76.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 76.560s; elapsed, 76.811s, #calls: 1', 'TOTAL : CPU, 76.560s; elapsed, 76.811s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 76.760s; elapsed, 76.833s, #calls: 1', 'TOTAL : CPU, 76.760s; elapsed, 76.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 76.720s; elapsed, 76.818s, #calls: 1', 'TOTAL : CPU, 76.720s; elapsed, 76.818s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 76.790s; elapsed, 76.820s, #calls: 1', 'TOTAL : CPU, 76.790s; elapsed, 76.820s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 214.860s; elapsed, 215.411s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.333s, #calls: 1', 'time for EIGEN_LDLT: CPU, 324.990s; elapsed, 325.138s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.820s; elapsed, 76.829s, #calls: 1', 'TOTAL : CPU, 617.000s; elapsed, 617.711s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 5040.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 218.540s; elapsed, 111.320s', 'individual call time for EIGEN_LDLT: CPU, 459.020s; elapsed, 459.216s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.311s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.23749', '# - matching time = 0.1441', '# - symmetrization time = 0.0726409', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.139036', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 98.7962', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.53864e+11', '# - flop rate = 7.26622 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.26672e+11', '# --------------------------------------------', '# total = 7.26672e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.04477e-11\trel.res = 1.92261e-15\tbw.error = 6.12855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.186963', '# - total flops = 2.86208e+08, min = 226725, max = 1.89071e+08', '# - flop rate = 1.53083 GFlop/s', '# - bytes moved = 434.988 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.3266 GByte/s', '# - solve arithmetic intensity = 0.657967 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 110.910s; elapsed, 100.713s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 138.940s; elapsed, 100.714s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 121.210s; elapsed, 100.714s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 193.100s; elapsed, 100.697s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 108.440s; elapsed, 100.691s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 109.010s; elapsed, 100.700s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 145.320s; elapsed, 100.690s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 101.680s; elapsed, 100.709s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 138.940s; elapsed, 100.714s, #calls: 1', 'TOTAL : CPU, 138.940s; elapsed, 100.714s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 110.910s; elapsed, 100.713s, #calls: 1', 'TOTAL : CPU, 110.910s; elapsed, 100.713s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 121.210s; elapsed, 100.714s, #calls: 1', 'TOTAL : CPU, 121.210s; elapsed, 100.714s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 108.440s; elapsed, 100.691s, #calls: 1', 'TOTAL : CPU, 108.440s; elapsed, 100.691s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 145.320s; elapsed, 100.690s, #calls: 1', 'TOTAL : CPU, 145.320s; elapsed, 100.690s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 193.100s; elapsed, 100.697s, #calls: 1', 'TOTAL : CPU, 193.100s; elapsed, 100.697s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 109.010s; elapsed, 100.700s, #calls: 1', 'TOTAL : CPU, 109.010s; elapsed, 100.700s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 218.540s; elapsed, 111.320s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.311s, #calls: 1', 'time for EIGEN_LDLT: CPU, 459.020s; elapsed, 459.216s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 101.680s; elapsed, 100.709s, #calls: 1', 'TOTAL : CPU, 779.700s; elapsed, 671.558s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 5040.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 487.180s; elapsed, 127.813s', 'individual call time for EIGEN_LDLT: CPU, 322.950s; elapsed, 322.890s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.245s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 130', '# - nd time = 1.2897', '# - matching time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.158281', '# - symmetrization time = 0.077889', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.140298', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 126.228', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.53864e+11', '# - flop rate = 5.68712 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.26672e+11', '# --------------------------------------------', '# total = 7.26672e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.16491e-11\trel.res = 1.95132e-15\tbw.error = 6.12855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.276265', '# - total flops = 2.86208e+08, min = 226725, max = 1.89071e+08', '# - flop rate = 1.03599 GFlop/s', '# - bytes moved = 435.006 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.5746 GByte/s', '# - solve arithmetic intensity = 0.65794 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 160.220s; elapsed, 128.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 226.790s; elapsed, 128.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 204.470s; elapsed, 128.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 151.980s; elapsed, 128.290s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 310.080s; elapsed, 128.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 153.530s; elapsed, 128.303s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 485.760s; elapsed, 128.302s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 131.570s; elapsed, 128.313s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 151.980s; elapsed, 128.290s, #calls: 1', 'TOTAL : CPU, 151.980s; elapsed, 128.290s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 310.080s; elapsed, 128.292s, #calls: 1', 'TOTAL : CPU, 310.080s; elapsed, 128.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 153.530s; elapsed, 128.303s, #calls: 1', 'TOTAL : CPU, 153.530s; elapsed, 128.303s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 160.220s; elapsed, 128.315s, #calls: 1', 'TOTAL : CPU, 160.220s; elapsed, 128.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 226.790s; elapsed, 128.315s, #calls: 1', 'TOTAL : CPU, 226.790s; elapsed, 128.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 204.470s; elapsed, 128.315s, #calls: 1', 'TOTAL : CPU, 204.470s; elapsed, 128.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 485.760s; elapsed, 128.302s, #calls: 1', 'TOTAL : CPU, 485.760s; elapsed, 128.302s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 487.180s; elapsed, 127.813s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.245s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.950s; elapsed, 322.890s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 131.570s; elapsed, 128.313s, #calls: 1', 'TOTAL : CPU, 942.220s; elapsed, 579.261s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 2520.0625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 234.480s; elapsed, 235.137s', 'individual call time for EIGEN_LDLT: CPU, 335.430s; elapsed, 335.733s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.331s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.319', '# - matching time = 0.15538', '# - symmetrization time = 0.0531712', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.130698', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 108.38', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.5344e+11', '# - flop rate = 6.62367 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.25118e+11', '# --------------------------------------------', '# total = 7.25118e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.05649e-11\trel.res = 1.68642e-15\tbw.error = 6.01998e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.146149', '# - total flops = 3.60242e+08, min = 214125, max = 1.8812e+08', '# - flop rate = 2.4649 GFlop/s', '# - bytes moved = 413.137 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.82682 GByte/s', '# - solve arithmetic intensity = 0.871968 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 110.190s; elapsed, 110.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 110.160s; elapsed, 110.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 110.040s; elapsed, 110.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 110.040s; elapsed, 110.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 109.740s; elapsed, 110.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 110.180s; elapsed, 110.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 110.190s; elapsed, 110.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 110.050s; elapsed, 110.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 110.180s; elapsed, 110.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 110.180s; elapsed, 110.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 110.140s; elapsed, 110.267s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 110.160s; elapsed, 110.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 110.180s; elapsed, 110.264s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 109.960s; elapsed, 110.265s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 110.150s; elapsed, 110.264s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 110.250s; elapsed, 110.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 109.740s; elapsed, 110.272s, #calls: 1', 'TOTAL : CPU, 109.740s; elapsed, 110.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 110.190s; elapsed, 110.277s, #calls: 1', 'TOTAL : CPU, 110.190s; elapsed, 110.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 110.160s; elapsed, 110.276s, #calls: 1', 'TOTAL : CPU, 110.160s; elapsed, 110.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 110.190s; elapsed, 110.266s, #calls: 1', 'TOTAL : CPU, 110.190s; elapsed, 110.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 110.180s; elapsed, 110.264s, #calls: 1', 'TOTAL : CPU, 110.180s; elapsed, 110.264s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 110.180s; elapsed, 110.276s, #calls: 1', 'TOTAL : CPU, 110.180s; elapsed, 110.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 110.040s; elapsed, 110.277s, #calls: 1', 'TOTAL : CPU, 110.040s; elapsed, 110.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 110.040s; elapsed, 110.277s, #calls: 1', 'TOTAL : CPU, 110.040s; elapsed, 110.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 110.180s; elapsed, 110.268s, #calls: 1', 'TOTAL : CPU, 110.180s; elapsed, 110.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 110.160s; elapsed, 110.271s, #calls: 1', 'TOTAL : CPU, 110.160s; elapsed, 110.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 109.960s; elapsed, 110.265s, #calls: 1', 'TOTAL : CPU, 109.960s; elapsed, 110.265s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 110.050s; elapsed, 110.276s, #calls: 1', 'TOTAL : CPU, 110.050s; elapsed, 110.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 110.150s; elapsed, 110.264s, #calls: 1', 'TOTAL : CPU, 110.150s; elapsed, 110.264s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 110.180s; elapsed, 110.268s, #calls: 1', 'TOTAL : CPU, 110.180s; elapsed, 110.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 110.140s; elapsed, 110.267s, #calls: 1', 'TOTAL : CPU, 110.140s; elapsed, 110.267s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 234.480s; elapsed, 235.137s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.331s, #calls: 1', 'time for EIGEN_LDLT: CPU, 335.430s; elapsed, 335.733s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 110.250s; elapsed, 110.279s, #calls: 1', 'TOTAL : CPU, 680.500s; elapsed, 681.479s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 2520.0625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 354.350s; elapsed, 184.489s', 'individual call time for EIGEN_LDLT: CPU, 335.890s; elapsed, 336.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.329s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.32772', '# - matching time = 0.160011', '# - symmetrization time = 0.062', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.15265', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 114.552', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.5344e+11', '# - flop rate = 6.2668 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.25118e+11', '# --------------------------------------------', '# total = 7.25118e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.04926e-11\trel.res = 1.68469e-15\tbw.error = 6.01998e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.143042', '# - total flops = 3.60242e+08, min = 214125, max = 1.8812e+08', '# - flop rate = 2.51844 GFlop/s', '# - bytes moved = 413.148 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.8883 GByte/s', '# - solve arithmetic intensity = 0.871946 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 123.940s; elapsed, 116.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 125.650s; elapsed, 116.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 219.690s; elapsed, 116.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 123.520s; elapsed, 116.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 122.630s; elapsed, 116.499s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 122.690s; elapsed, 116.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 122.000s; elapsed, 116.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 185.610s; elapsed, 116.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 124.900s; elapsed, 116.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 124.660s; elapsed, 116.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 152.540s; elapsed, 116.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 139.030s; elapsed, 116.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 140.850s; elapsed, 116.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 124.110s; elapsed, 116.497s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 123.690s; elapsed, 116.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 117.410s; elapsed, 116.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 122.690s; elapsed, 116.493s, #calls: 1', 'TOTAL : CPU, 122.690s; elapsed, 116.493s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 185.610s; elapsed, 116.493s, #calls: 1', 'TOTAL : CPU, 185.610s; elapsed, 116.493s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 122.630s; elapsed, 116.499s, #calls: 1', 'TOTAL : CPU, 122.630s; elapsed, 116.499s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 140.850s; elapsed, 116.503s, #calls: 1', 'TOTAL : CPU, 140.850s; elapsed, 116.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 123.520s; elapsed, 116.496s, #calls: 1', 'TOTAL : CPU, 123.520s; elapsed, 116.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 123.690s; elapsed, 116.496s, #calls: 1', 'TOTAL : CPU, 123.690s; elapsed, 116.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 122.000s; elapsed, 116.491s, #calls: 1', 'TOTAL : CPU, 122.000s; elapsed, 116.491s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 124.900s; elapsed, 116.492s, #calls: 1', 'TOTAL : CPU, 124.900s; elapsed, 116.492s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 152.540s; elapsed, 116.503s, #calls: 1', 'TOTAL : CPU, 152.540s; elapsed, 116.503s', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 124.110s; elapsed, 116.497s, #calls: 1', 'TOTAL : CPU, 124.110s; elapsed, 116.497s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 219.690s; elapsed, 116.496s, #calls: 1', 'TOTAL : CPU, 219.690s; elapsed, 116.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 125.650s; elapsed, 116.503s, #calls: 1', 'TOTAL : CPU, 125.650s; elapsed, 116.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 123.940s; elapsed, 116.503s, #calls: 1', 'TOTAL : CPU, 123.940s; elapsed, 116.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 124.660s; elapsed, 116.502s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 139.030s; elapsed, 116.503s, #calls: 1', 'TOTAL : CPU, 139.030s; elapsed, 116.503s', 'TOTAL : CPU, 124.660s; elapsed, 116.502s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 354.350s; elapsed, 184.489s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.329s, #calls: 1', 'time for EIGEN_LDLT: CPU, 335.890s; elapsed, 336.047s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 117.410s; elapsed, 116.506s, #calls: 1', 'TOTAL : CPU, 808.140s; elapsed, 637.371s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.160s; elapsed, 146.299s', 'individual call time for EIGEN_LDLT: CPU, 292.730s; elapsed, 292.927s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.271s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.160s; elapsed, 146.299s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.271s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.730s; elapsed, 292.927s, #calls: 1', 'TOTAL : CPU, 439.160s; elapsed, 439.497s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 233.900s; elapsed, 118.604s', 'individual call time for EIGEN_LDLT: CPU, 293.120s; elapsed, 293.117s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.273s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 233.900s; elapsed, 118.604s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.273s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.120s; elapsed, 293.117s, #calls: 1', 'TOTAL : CPU, 527.420s; elapsed, 411.994s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 402.440s; elapsed, 103.778s', 'individual call time for EIGEN_LDLT: CPU, 322.990s; elapsed, 322.935s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.263s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 402.440s; elapsed, 103.778s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.263s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.990s; elapsed, 322.935s, #calls: 1', 'TOTAL : CPU, 725.940s; elapsed, 426.976s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 851.390s; elapsed, 109.176s', 'individual call time for EIGEN_LDLT: CPU, 322.500s; elapsed, 322.205s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.670s; elapsed, 0.224s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 851.390s; elapsed, 109.176s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.670s; elapsed, 0.224s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.500s; elapsed, 322.205s, #calls: 1', 'TOTAL : CPU, 1174.560s; elapsed, 431.605s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2016.960s; elapsed, 128.383s', 'individual call time for EIGEN_LDLT: CPU, 346.070s; elapsed, 345.610s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.110s; elapsed, 0.241s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2016.960s; elapsed, 128.383s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.110s; elapsed, 0.241s, #calls: 1', 'time for EIGEN_LDLT: CPU, 346.070s; elapsed, 345.610s, #calls: 1', 'TOTAL : CPU, 2364.140s; elapsed, 474.234s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 145.730s; elapsed, 145.857s', 'individual call time for EIGEN_LDLT: CPU, 292.800s; elapsed, 292.993s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.270s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# - nd time = 1.13388', '# - matching time = 0.12792', '# - symmetrization time = 0.100112', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.154529', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 158.716', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18415e+11, min = 41770, max = 7.18415e+11', '# - flop rate = 4.52641 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.20396e-10\trel.res = 2.87732e-15\tbw.error = 7.77224e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.427642', '# - total flops = 2.36199e+08, min = 177542, max = 2.36022e+08', '# - flop rate = 0.55233 GFlop/s', '# - bytes moved = 1221.56 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.85651 GByte/s', '# - solve arithmetic intensity = 0.193358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 160.730s; elapsed, 160.898s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 160.880s; elapsed, 160.894s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 160.730s; elapsed, 160.898s, #calls: 1', 'TOTAL : CPU, 160.730s; elapsed, 160.898s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 145.730s; elapsed, 145.857s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.270s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.800s; elapsed, 292.993s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 160.880s; elapsed, 160.894s, #calls: 1', 'TOTAL : CPU, 599.670s; elapsed, 600.014s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 239.420s; elapsed, 121.517s', 'individual call time for EIGEN_LDLT: CPU, 292.760s; elapsed, 292.765s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.253s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.18001', '# - matching time = 0.13027', '# - symmetrization time = 0.101257', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.150681', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 107.948', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 41770, max = 7.18416e+11', '# - flop rate = 6.65523 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1636e-10\trel.res = 2.78086e-15\tbw.error = 7.76343e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.295489', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.799669 GFlop/s', '# - bytes moved = 1223.99 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.14226 GByte/s', '# - solve arithmetic intensity = 0.193051 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 214.900s; elapsed, 110.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.230s; elapsed, 110.045s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 214.900s; elapsed, 110.049s, #calls: 1', 'TOTAL : CPU, 214.900s; elapsed, 110.049s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 239.420s; elapsed, 121.517s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.253s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.760s; elapsed, 292.765s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 111.230s; elapsed, 110.045s, #calls: 1', 'TOTAL : CPU, 643.770s; elapsed, 524.580s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 525.780s; elapsed, 135.402s', 'individual call time for EIGEN_LDLT: CPU, 322.690s; elapsed, 322.648s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.273s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.331', '# - matching time = 0.158291', '# - symmetrization time = 0.110958', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.166213', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 72.1254', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 41770, max = 7.18416e+11', '# - flop rate = 9.96065 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15759e-10\trel.res = 2.7665e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.349003', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.677052 GFlop/s', '# - bytes moved = 1225.24 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.51068 GByte/s', '# - solve arithmetic intensity = 0.192855 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 286.740s; elapsed, 74.530s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 78.490s; elapsed, 74.520s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 286.740s; elapsed, 74.530s, #calls: 1', 'TOTAL : CPU, 286.740s; elapsed, 74.530s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 525.780s; elapsed, 135.402s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.273s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.690s; elapsed, 322.648s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 78.490s; elapsed, 74.520s, #calls: 1', 'TOTAL : CPU, 927.480s; elapsed, 532.844s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1012.020s; elapsed, 129.361s', 'individual call time for EIGEN_LDLT: CPU, 326.410s; elapsed, 326.213s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.239s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.20704', '# - matching time = 0.152491', '# - symmetrization time = 0.11131', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.157382', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 92.4916', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 42535, max = 7.18416e+11', '# - flop rate = 7.76737 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15744e-10\trel.res = 2.76614e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.43548', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.542604 GFlop/s', '# - bytes moved = 1225.31 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.8137 GByte/s', '# - solve arithmetic intensity = 0.192844 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 734.780s; elapsed, 94.861s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 104.660s; elapsed, 94.856s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 734.780s; elapsed, 94.861s, #calls: 1', 'TOTAL : CPU, 734.780s; elapsed, 94.861s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1012.020s; elapsed, 129.361s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 326.410s; elapsed, 326.213s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 104.660s; elapsed, 94.856s, #calls: 1', 'TOTAL : CPU, 1443.800s; elapsed, 550.670s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1792.880s; elapsed, 114.528s', 'individual call time for EIGEN_LDLT: CPU, 317.850s; elapsed, 316.228s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.160s; elapsed, 0.221s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.25031', '# - matching time = 0.165405', '# - symmetrization time = 0.168835', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.18917', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 124.356', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.1842e+11, min = 43208, max = 7.1842e+11', '# - flop rate = 5.77714 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15728e-10\trel.res = 2.76578e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.646924', '# - total flops = 2.36302e+08, min = 177542, max = 2.36125e+08', '# - flop rate = 0.365271 GFlop/s', '# - bytes moved = 1225.6 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.8945 GByte/s', '# - solve arithmetic intensity = 0.192806 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1985.570s; elapsed, 127.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 149.540s; elapsed, 127.086s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1985.570s; elapsed, 127.101s, #calls: 1', 'TOTAL : CPU, 1985.570s; elapsed, 127.101s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1792.880s; elapsed, 114.528s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.160s; elapsed, 0.221s, #calls: 1', 'time for EIGEN_LDLT: CPU, 317.850s; elapsed, 316.228s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 149.540s; elapsed, 127.086s, #calls: 1', 'TOTAL : CPU, 2261.430s; elapsed, 558.063s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 151.850s; elapsed, 152.012s', 'individual call time for EIGEN_LDLT: CPU, 301.960s; elapsed, 302.143s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.283s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.19436', '# - matching time = 0.134607', '# - symmetrization time = 0.0979528', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.107145', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 95.1278', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 7.56906 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10513e-10\trel.res = 2.64113e-15\tbw.error = 7.89018e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.269647', '# - total flops = 2.52082e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 0.93486 GFlop/s', '# - bytes moved = 460.028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.70604 GByte/s', '# - solve arithmetic intensity = 0.547971 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 96.950s; elapsed, 97.075s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 96.810s; elapsed, 97.048s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 96.860s; elapsed, 97.042s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 97.050s; elapsed, 97.060s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 96.950s; elapsed, 97.075s, #calls: 1', 'TOTAL : CPU, 96.950s; elapsed, 97.075s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 96.810s; elapsed, 97.048s, #calls: 1', 'TOTAL : CPU, 96.810s; elapsed, 97.048s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 96.860s; elapsed, 97.042s, #calls: 1', 'TOTAL : CPU, 96.860s; elapsed, 97.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 151.850s; elapsed, 152.012s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.283s, #calls: 1', 'time for EIGEN_LDLT: CPU, 301.960s; elapsed, 302.143s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 97.050s; elapsed, 97.060s, #calls: 1', 'TOTAL : CPU, 551.140s; elapsed, 551.499s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 234.400s; elapsed, 119.342s', 'individual call time for EIGEN_LDLT: CPU, 658.530s; elapsed, 658.683s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.395s', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2CPP -22 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '', '', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.12799', '# - matching time = 0.142352', '# - symmetrization time = 0.10072', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.120242', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 87.3635', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 8.24174 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10477e-10\trel.res = 2.64028e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.26661', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 0.94557 GFlop/s', '# - bytes moved = 460.383 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.7268 GByte/s', '# - solve arithmetic intensity = 0.547585 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 134.770s; elapsed, 89.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 174.320s; elapsed, 89.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 135.630s; elapsed, 89.252s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 90.310s; elapsed, 89.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 135.630s; elapsed, 89.252s, #calls: 1', 'TOTAL : CPU, 135.630s; elapsed, 89.252s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 134.770s; elapsed, 89.284s, #calls: 1', 'TOTAL : CPU, 134.770s; elapsed, 89.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 174.320s; elapsed, 89.258s, #calls: 1', 'TOTAL : CPU, 174.320s; elapsed, 89.258s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 234.400s; elapsed, 119.342s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.395s, #calls: 1', 'time for EIGEN_LDLT: CPU, 658.530s; elapsed, 658.683s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 90.310s; elapsed, 89.268s, #calls: 1', 'TOTAL : CPU, 983.870s; elapsed, 867.689s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 466.920s; elapsed, 121.914s', 'individual call time for EIGEN_LDLT: CPU, 328.840s; elapsed, 328.851s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.590s; elapsed, 0.281s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.22233', '# - matching time = 0.143358', '# - symmetrization time = 0.098588', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.131311', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 93.5173', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 7.6994 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1046e-10\trel.res = 2.63986e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.277433', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 0.908682 GFlop/s', '# - bytes moved = 460.395 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.65948 GByte/s', '# - solve arithmetic intensity = 0.54757 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 215.280s; elapsed, 95.551s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 260.740s; elapsed, 95.519s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 370.340s; elapsed, 95.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 98.790s; elapsed, 95.537s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 260.740s; elapsed, 95.519s, #calls: 1', 'TOTAL : CPU, 260.740s; elapsed, 95.519s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 370.340s; elapsed, 95.524s, #calls: 1', 'TOTAL : CPU, 370.340s; elapsed, 95.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 215.280s; elapsed, 95.551s, #calls: 1', 'TOTAL : CPU, 215.280s; elapsed, 95.551s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 466.920s; elapsed, 121.914s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.590s; elapsed, 0.281s, #calls: 1', 'time for EIGEN_LDLT: CPU, 328.840s; elapsed, 328.851s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 98.790s; elapsed, 95.537s, #calls: 1', 'TOTAL : CPU, 895.140s; elapsed, 546.583s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 871.570s; elapsed, 111.770s', 'individual call time for EIGEN_LDLT: CPU, 326.580s; elapsed, 326.221s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.760s; elapsed, 0.243s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.20507', '# - matching time = 0.162998', '# - symmetrization time = 0.110007', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.173188', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 97.06', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 42535, max = 5.52422e+11', '# - flop rate = 7.41837 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10436e-10\trel.res = 2.6393e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.4315', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 0.584238 GFlop/s', '# - bytes moved = 460.421 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.06702 GByte/s', '# - solve arithmetic intensity = 0.54754 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 456.460s; elapsed, 99.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 457.710s; elapsed, 99.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 759.310s; elapsed, 99.294s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 108.090s; elapsed, 99.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 456.460s; elapsed, 99.318s, #calls: 1', 'TOTAL : CPU, 456.460s; elapsed, 99.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 457.710s; elapsed, 99.284s, #calls: 1', 'TOTAL : CPU, 457.710s; elapsed, 99.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 759.310s; elapsed, 99.294s, #calls: 1', 'TOTAL : CPU, 759.310s; elapsed, 99.294s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 871.570s; elapsed, 111.770s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.760s; elapsed, 0.243s, #calls: 1', 'time for EIGEN_LDLT: CPU, 326.580s; elapsed, 326.221s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 108.090s; elapsed, 99.305s, #calls: 1', 'TOTAL : CPU, 1307.000s; elapsed, 537.539s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 252.050s; elapsed, 252.704s', 'individual call time for EIGEN_LDLT: CPU, 332.800s; elapsed, 333.002s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.349s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.30776', '# - matching time = 0.140348', '# - symmetrization time = 0.0674729', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.124222', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 113.058', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 6.36641 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.24362e-11\trel.res = 1.97013e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.212895', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.35312 GFlop/s', '# - bytes moved = 434.79 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04227 GByte/s', '# - solve arithmetic intensity = 0.662556 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 114.890s; elapsed, 115.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 114.980s; elapsed, 115.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 114.690s; elapsed, 115.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 114.970s; elapsed, 115.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 114.950s; elapsed, 115.014s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 114.980s; elapsed, 115.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 115.000s; elapsed, 115.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.020s; elapsed, 115.033s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 114.690s; elapsed, 115.022s, #calls: 1', 'TOTAL : CPU, 114.690s; elapsed, 115.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 115.000s; elapsed, 115.022s, #calls: 1', 'TOTAL : CPU, 115.000s; elapsed, 115.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 114.950s; elapsed, 115.014s, #calls: 1', 'TOTAL : CPU, 114.950s; elapsed, 115.014s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 114.970s; elapsed, 115.021s, #calls: 1', 'TOTAL : CPU, 114.970s; elapsed, 115.021s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 114.980s; elapsed, 115.035s, #calls: 1', 'TOTAL : CPU, 114.980s; elapsed, 115.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 114.980s; elapsed, 115.021s, #calls: 1', 'TOTAL : CPU, 114.980s; elapsed, 115.021s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 114.890s; elapsed, 115.035s, #calls: 1', 'TOTAL : CPU, 114.890s; elapsed, 115.035s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 252.050s; elapsed, 252.704s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.349s, #calls: 1', 'time for EIGEN_LDLT: CPU, 332.800s; elapsed, 333.002s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.020s; elapsed, 115.033s, #calls: 1', 'TOTAL : CPU, 700.210s; elapsed, 701.088s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 238.010s; elapsed, 121.249s', 'individual call time for EIGEN_LDLT: CPU, 320.500s; elapsed, 320.574s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.295s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', 'CPP -2', 'CPP -2', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.34453', '# - matching time = 0.156378', '# - symmetrization time = 0.0744369', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.157503', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 109.179', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 6.59261 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.22963e-11\trel.res = 1.96679e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.192903', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.49335 GFlop/s', '# - bytes moved = 434.803 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.254 GByte/s', '# - solve arithmetic intensity = 0.662536 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 120.050s; elapsed, 111.219s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 122.040s; elapsed, 111.221s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 214.640s; elapsed, 111.221s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 120.630s; elapsed, 111.223s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 138.880s; elapsed, 111.246s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 134.920s; elapsed, 111.247s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 182.170s; elapsed, 111.223s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 112.240s; elapsed, 111.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 120.050s; elapsed, 111.219s, #calls: 1', 'TOTAL : CPU, 120.050s; elapsed, 111.219s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 122.040s; elapsed, 111.221s, #calls: 1', 'TOTAL : CPU, 122.040s; elapsed, 111.221s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 138.880s; elapsed, 111.246s, #calls: 1', 'TOTAL : CPU, 138.880s; elapsed, 111.246s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 120.630s; elapsed, 111.223s, #calls: 1', 'TOTAL : CPU, 120.630s; elapsed, 111.223s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 214.640s; elapsed, 111.221s, #calls: 1', 'TOTAL : CPU, 214.640s; elapsed, 111.221s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 134.920s; elapsed, 111.247s, #calls: 1', 'TOTAL : CPU, 134.920s; elapsed, 111.247s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 182.170s; elapsed, 111.223s, #calls: 1', 'TOTAL : CPU, 182.170s; elapsed, 111.223s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 238.010s; elapsed, 121.249s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.295s, #calls: 1', 'time for EIGEN_LDLT: CPU, 320.500s; elapsed, 320.574s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 112.240s; elapsed, 111.241s, #calls: 1', 'TOTAL : CPU, 671.200s; elapsed, 553.358s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 514.140s; elapsed, 130.364s', 'individual call time for EIGEN_LDLT: CPU, 332.180s; elapsed, 332.279s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.580s; elapsed, 0.290s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.38205', '# - matching time = 0.156389', '# - symmetrization time = 0.074542', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.137639', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 104.47', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 6.88977 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.22801e-11\trel.res = 1.9664e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.292801', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 0.983851 GFlop/s', '# - bytes moved = 434.821 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.48504 GByte/s', '# - solve arithmetic intensity = 0.662508 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 226.340s; elapsed, 106.653s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 174.830s; elapsed, 106.652s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 135.360s; elapsed, 106.639s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 241.880s; elapsed, 106.633s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 130.490s; elapsed, 106.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 129.240s; elapsed, 106.638s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 403.640s; elapsed, 106.638s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 109.920s; elapsed, 106.649s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 241.880s; elapsed, 106.633s, #calls: 1', 'TOTAL : CPU, 241.880s; elapsed, 106.633s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 135.360s; elapsed, 106.639s, #calls: 1', 'TOTAL : CPU, 135.360s; elapsed, 106.639s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 129.240s; elapsed, 106.638s, #calls: 1', 'TOTAL : CPU, 129.240s; elapsed, 106.638s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 130.490s; elapsed, 106.640s, #calls: 1', 'TOTAL : CPU, 130.490s; elapsed, 106.640s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 226.340s; elapsed, 106.653s, #calls: 1', 'TOTAL : CPU, 226.340s; elapsed, 106.653s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 174.830s; elapsed, 106.652s, #calls: 1', 'TOTAL : CPU, 174.830s; elapsed, 106.652s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 403.640s; elapsed, 106.638s, #calls: 1', 'TOTAL : CPU, 403.640s; elapsed, 106.638s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 514.140s; elapsed, 130.364s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.580s; elapsed, 0.290s, #calls: 1', 'time for EIGEN_LDLT: CPU, 332.180s; elapsed, 332.279s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 109.920s; elapsed, 106.649s, #calls: 1', 'TOTAL : CPU, 956.820s; elapsed, 569.581s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 243.360s; elapsed, 243.951s', 'individual call time for EIGEN_LDLT: CPU, 335.410s; elapsed, 335.694s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.342s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 127', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.24979', '# - matching time = 0.140896', '# - symmetrization time = 0.0516391', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.13182', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 96.3026', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51438e+11', '# - flop rate = 7.47409 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.27001e+11', '# --------------------------------------------', '# total = 7.27001e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.95624e-11\trel.res = 2.14044e-15\tbw.error = 6.17961e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.153332', '# - total flops = 3.64315e+08, min = 100280, max = 1.87793e+08', '# - flop rate = 2.37599 GFlop/s', '# - bytes moved = 412.812 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.69228 GByte/s', '# - solve arithmetic intensity = 0.882521 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 98.030s; elapsed, 98.125s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 98.040s; elapsed, 98.118s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 98.040s; elapsed, 98.116s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 98.050s; elapsed, 98.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 98.070s; elapsed, 98.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 98.060s; elapsed, 98.119s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 98.090s; elapsed, 98.113s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 97.780s; elapsed, 98.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 98.070s; elapsed, 98.116s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 98.040s; elapsed, 98.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 98.090s; elapsed, 98.119s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 97.890s; elapsed, 98.125s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 97.980s; elapsed, 98.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 98.010s; elapsed, 98.125s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 97.840s; elapsed, 98.119s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 98.130s; elapsed, 98.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 97.980s; elapsed, 98.126s, #calls: 1', 'TOTAL : CPU, 97.980s; elapsed, 98.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 98.070s; elapsed, 98.116s, #calls: 1', 'TOTAL : CPU, 98.070s; elapsed, 98.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 98.030s; elapsed, 98.125s, #calls: 1', 'TOTAL : CPU, 98.030s; elapsed, 98.125s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 97.840s; elapsed, 98.119s, #calls: 1', 'TOTAL : CPU, 97.840s; elapsed, 98.119s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 98.090s; elapsed, 98.119s, #calls: 1', 'TOTAL : CPU, 98.090s; elapsed, 98.119s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 98.070s; elapsed, 98.120s, #calls: 1', 'TOTAL : CPU, 98.070s; elapsed, 98.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 98.040s; elapsed, 98.118s, #calls: 1', 'TOTAL : CPU, 98.040s; elapsed, 98.118s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 97.890s; elapsed, 98.125s, #calls: 1', 'TOTAL : CPU, 97.890s; elapsed, 98.125s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 97.780s; elapsed, 98.114s, #calls: 1', 'TOTAL : CPU, 97.780s; elapsed, 98.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 98.040s; elapsed, 98.114s, #calls: 1', 'TOTAL : CPU, 98.040s; elapsed, 98.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 98.040s; elapsed, 98.116s, #calls: 1', 'TOTAL : CPU, 98.040s; elapsed, 98.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 98.060s; elapsed, 98.119s, #calls: 1', 'TOTAL : CPU, 98.060s; elapsed, 98.119s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 98.010s; elapsed, 98.125s, #calls: 1', 'TOTAL : CPU, 98.010s; elapsed, 98.125s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 98.090s; elapsed, 98.113s, #calls: 1', 'TOTAL : CPU, 98.090s; elapsed, 98.113s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 98.050s; elapsed, 98.117s, #calls: 1', 'TOTAL : CPU, 98.050s; elapsed, 98.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 243.360s; elapsed, 243.951s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.342s, #calls: 1', 'time for EIGEN_LDLT: CPU, 335.410s; elapsed, 335.694s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 98.130s; elapsed, 98.127s, #calls: 1', 'TOTAL : CPU, 677.240s; elapsed, 678.113s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 267.150s; elapsed, 135.355s', 'individual call time for EIGEN_LDLT: CPU, 336.150s; elapsed, 336.308s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.347s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 127', '# - nd time = 1.2593', '# - matching time = 0.157941', '# - symmetrization time = 0.0584149', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.14598', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 115.472', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51438e+11', '# - flop rate = 6.2333 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.27001e+11', '# --------------------------------------------', '# total = 7.27001e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.94628e-11\trel.res = 2.13806e-15\tbw.error = 6.1844e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.139836', '# - total flops = 3.64315e+08, min = 100280, max = 1.87793e+08', '# - flop rate = 2.60531 GFlop/s', '# - bytes moved = 412.822 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.95219 GByte/s', '# - solve arithmetic intensity = 0.882499 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 137.130s; elapsed, 117.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 125.660s; elapsed, 117.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 139.870s; elapsed, 117.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 153.330s; elapsed, 117.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 124.050s; elapsed, 117.337s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 123.120s; elapsed, 117.337s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 125.110s; elapsed, 117.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 124.760s; elapsed, 117.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 222.510s; elapsed, 117.337s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 124.670s; elapsed, 117.337s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 124.180s; elapsed, 117.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 123.870s; elapsed, 117.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 122.990s; elapsed, 117.337s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 122.470s; elapsed, 117.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 175.930s; elapsed, 117.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.250s; elapsed, 117.345s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 124.670s; elapsed, 117.337s, #calls: 1', 'TOTAL : CPU, 124.670s; elapsed, 117.337s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 123.870s; elapsed, 117.334s, #calls: 1', 'TOTAL : CPU, 123.870s; elapsed, 117.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 125.660s; elapsed, 117.343s, #calls: 1', 'TOTAL : CPU, 125.660s; elapsed, 117.343s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 137.130s; elapsed, 117.341s, #calls: 1', 'TOTAL : CPU, 137.130s; elapsed, 117.341s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 123.120s; elapsed, 117.337s, #calls: 1', 'TOTAL : CPU, 123.120s; elapsed, 117.337s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 124.180s; elapsed, 117.334s, #calls: 1', 'TOTAL : CPU, 124.180s; elapsed, 117.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 222.510s; elapsed, 117.337s, #calls: 1', 'TOTAL : CPU, 222.510s; elapsed, 117.337s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 124.050s; elapsed, 117.337s, #calls: 1', 'TOTAL : CPU, 124.050s; elapsed, 117.337s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 122.470s; elapsed, 117.331s, #calls: 1', 'TOTAL : CPU, 122.470s; elapsed, 117.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 124.760s; elapsed, 117.332s, #calls: 1', 'TOTAL : CPU, 124.760s; elapsed, 117.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 153.330s; elapsed, 117.342s, #calls: 1', 'TOTAL : CPU, 153.330s; elapsed, 117.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 122.990s; elapsed, 117.337s, #calls: 1', 'TOTAL : CPU, 122.990s; elapsed, 117.337s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 139.870s; elapsed, 117.342s, #calls: 1', 'TOTAL : CPU, 139.870s; elapsed, 117.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 125.110s; elapsed, 117.332s, #calls: 1', 'TOTAL : CPU, 125.110s; elapsed, 117.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 175.930s; elapsed, 117.330s, #calls: 1', 'TOTAL : CPU, 175.930s; elapsed, 117.330s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 267.150s; elapsed, 135.355s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.347s, #calls: 1', 'time for EIGEN_LDLT: CPU, 336.150s; elapsed, 336.308s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.250s; elapsed, 117.345s, #calls: 1', 'TOTAL : CPU, 722.050s; elapsed, 589.354s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.370s; elapsed, 31.391s', 'individual call time for EIGEN_LDLT: CPU, 37.640s; elapsed, 37.669s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.120s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.370s; elapsed, 31.391s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.640s; elapsed, 37.669s, #calls: 1', 'TOTAL : CPU, 69.130s; elapsed, 69.180s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 42.030s; elapsed, 21.430s', 'individual call time for EIGEN_LDLT: CPU, 37.930s; elapsed, 37.781s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.124s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 42.030s; elapsed, 21.430s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.124s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.930s; elapsed, 37.781s, #calls: 1', 'TOTAL : CPU, 80.140s; elapsed, 59.336s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 92.750s; elapsed, 23.754s', 'individual call time for EIGEN_LDLT: CPU, 38.220s; elapsed, 37.770s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.113s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 92.750s; elapsed, 23.754s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.220s; elapsed, 37.770s, #calls: 1', 'TOTAL : CPU, 131.200s; elapsed, 61.636s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 192.450s; elapsed, 24.788s', 'individual call time for EIGEN_LDLT: CPU, 38.770s; elapsed, 37.736s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.103s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 192.450s; elapsed, 24.788s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.103s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.770s; elapsed, 37.736s, #calls: 1', 'TOTAL : CPU, 231.490s; elapsed, 62.627s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 398.020s; elapsed, 25.399s', 'individual call time for EIGEN_LDLT: CPU, 43.870s; elapsed, 41.693s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.580s; elapsed, 0.107s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 398.020s; elapsed, 25.399s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.580s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 43.870s; elapsed, 41.693s, #calls: 1', 'TOTAL : CPU, 442.470s; elapsed, 67.198s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.850s; elapsed, 32.884s', 'individual call time for EIGEN_LDLT: CPU, 39.030s; elapsed, 39.063s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.406928', '# - matching time = 0.0722132', '# - symmetrization time = 0.0364408', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0853631', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 30.3013', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48564e+10, min = 222880, max = 9.48562e+10', '# - flop rate = 3.13044 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.11914e-11\trel.res = 2.20224e-15\tbw.error = 4.90384e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.11235', '# - total flops = 6.78544e+07, min = 289862, max = 6.75645e+07', '# - flop rate = 0.603956 GFlop/s', '# - bytes moved = 410.995 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.65816 GByte/s', '# - solve arithmetic intensity = 0.165098 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.090s; elapsed, 31.111s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.110s; elapsed, 31.104s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.090s; elapsed, 31.111s, #calls: 1', 'TOTAL : CPU, 31.090s; elapsed, 31.111s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.850s; elapsed, 32.884s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.030s; elapsed, 39.063s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.110s; elapsed, 31.104s, #calls: 1', 'TOTAL : CPU, 103.120s; elapsed, 103.179s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 43.150s; elapsed, 22.039s', 'individual call time for EIGEN_LDLT: CPU, 37.880s; elapsed, 37.722s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.124s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.397167', '# - matching time = 0.067879', '# - symmetrization time = 0.035713', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0992429', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 23.8972', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 222880, max = 9.48565e+10', '# - flop rate = 3.96936 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04299e-11\trel.res = 2.18385e-15\tbw.error = 4.87162e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.104128', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.65183 GFlop/s', '# - bytes moved = 411.788 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.95464 GByte/s', '# - solve arithmetic intensity = 0.164827 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 48.780s; elapsed, 24.720s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 25.470s; elapsed, 24.714s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 48.780s; elapsed, 24.720s, #calls: 1', 'TOTAL : CPU, 48.780s; elapsed, 24.720s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 43.150s; elapsed, 22.039s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.124s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.880s; elapsed, 37.722s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 25.470s; elapsed, 24.714s, #calls: 1', 'TOTAL : CPU, 106.680s; elapsed, 84.599s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 104.970s; elapsed, 27.213s', 'individual call time for EIGEN_LDLT: CPU, 39.610s; elapsed, 39.189s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.132s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.419348', '# - matching time = 0.073046', '# - symmetrization time = 0.038209', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084626', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 16.2723', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 222880, max = 9.48565e+10', '# - flop rate = 5.82935 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04185e-11\trel.res = 2.18357e-15\tbw.error = 4.87162e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.155805', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.435633 GFlop/s', '# - bytes moved = 411.808 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.6431 GByte/s', '# - solve arithmetic intensity = 0.164819 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.670s; elapsed, 17.166s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.590s; elapsed, 17.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.670s; elapsed, 17.166s, #calls: 1', 'TOTAL : CPU, 66.670s; elapsed, 17.166s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 104.970s; elapsed, 27.213s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.132s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.610s; elapsed, 39.189s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.590s; elapsed, 17.159s, #calls: 1', 'TOTAL : CPU, 164.430s; elapsed, 83.693s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 174.260s; elapsed, 22.421s', 'individual call time for EIGEN_LDLT: CPU, 42.970s; elapsed, 41.914s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.114s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.403873', '# - matching time = 0.0838101', '# - symmetrization time = 0.042819', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.095258', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 21.4472', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 224830, max = 9.48565e+10', '# - flop rate = 4.4228 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04226e-11\trel.res = 2.18367e-15\tbw.error = 4.86687e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.144882', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.468476 GFlop/s', '# - bytes moved = 411.848 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.84264 GByte/s', '# - solve arithmetic intensity = 0.164803 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 174.130s; elapsed, 22.349s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.130s; elapsed, 22.341s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 174.130s; elapsed, 22.349s, #calls: 1', 'TOTAL : CPU, 174.130s; elapsed, 22.349s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 174.260s; elapsed, 22.421s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.114s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.970s; elapsed, 41.914s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.130s; elapsed, 22.341s, #calls: 1', 'TOTAL : CPU, 245.650s; elapsed, 86.790s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 258.880s; elapsed, 16.712s', 'individual call time for EIGEN_LDLT: CPU, 39.960s; elapsed, 37.746s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.414389', '# - matching time = 0.080286', '# - symmetrization time = 0.045187', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.109492', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 16.005', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48577e+10, min = 226855, max = 9.48575e+10', '# - flop rate = 5.92674 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04391e-11\trel.res = 2.18407e-15\tbw.error = 4.87578e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.132886', '# - total flops = 6.78784e+07, min = 289862, max = 6.75885e+07', '# - flop rate = 0.510801 GFlop/s', '# - bytes moved = 412.003 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.10042 GByte/s', '# - solve arithmetic intensity = 0.164752 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 263.320s; elapsed, 16.930s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 29.520s; elapsed, 16.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 263.320s; elapsed, 16.930s, #calls: 1', 'TOTAL : CPU, 263.320s; elapsed, 16.930s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 258.880s; elapsed, 16.712s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.960s; elapsed, 37.746s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 29.520s; elapsed, 16.926s, #calls: 1', 'TOTAL : CPU, 328.830s; elapsed, 71.480s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.820s; elapsed, 32.844s', 'individual call time for EIGEN_LDLT: CPU, 38.850s; elapsed, 38.883s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.485402', '# - matching time = 0.0721219', '# - symmetrization time = 0.0354371', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.061687', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 16.6704', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 5.6669 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00117e-10\trel.res = 2.4178e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.080785', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.885471 GFlop/s', '# - bytes moved = 219.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71757 GByte/s', '# - solve arithmetic intensity = 0.325832 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.410s; elapsed, 17.474s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.420s; elapsed, 17.463s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.430s; elapsed, 17.457s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.470s; elapsed, 17.471s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.420s; elapsed, 17.463s, #calls: 1', 'TOTAL : CPU, 17.420s; elapsed, 17.463s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.430s; elapsed, 17.457s, #calls: 1', 'TOTAL : CPU, 17.430s; elapsed, 17.457s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.410s; elapsed, 17.474s, #calls: 1', 'TOTAL : CPU, 17.410s; elapsed, 17.474s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.820s; elapsed, 32.844s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.850s; elapsed, 38.883s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.470s; elapsed, 17.471s, #calls: 1', 'TOTAL : CPU, 89.260s; elapsed, 89.323s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 44.100s; elapsed, 22.544s', 'individual call time for EIGEN_LDLT: CPU, 40.240s; elapsed, 40.091s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.520953', '# - matching time = 0.084276', '# - symmetrization time = 0.036906', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0712121', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 20.259', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 4.66311 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.99927e-11\trel.res = 2.41479e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.107443', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.665774 GFlop/s', '# - bytes moved = 219.551 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04342 GByte/s', '# - solve arithmetic intensity = 0.325814 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 30.410s; elapsed, 21.157s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 33.480s; elapsed, 21.145s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.500s; elapsed, 21.140s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.870s; elapsed, 21.152s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.500s; elapsed, 21.140s, #calls: 1', 'TOTAL : CPU, 41.500s; elapsed, 21.140s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 33.480s; elapsed, 21.145s, #calls: 1', 'TOTAL : CPU, 33.480s; elapsed, 21.145s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 30.410s; elapsed, 21.157s, #calls: 1', 'TOTAL : CPU, 30.410s; elapsed, 21.157s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 44.100s; elapsed, 22.544s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.240s; elapsed, 40.091s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.870s; elapsed, 21.152s, #calls: 1', 'TOTAL : CPU, 106.380s; elapsed, 83.911s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 89.460s; elapsed, 23.088s', 'individual call time for EIGEN_LDLT: CPU, 39.400s; elapsed, 38.877s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.122s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.491113', '# - matching time = 0.0742049', '# - symmetrization time = 0.036206', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0757251', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 21.4312', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 4.40805 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.99807e-11\trel.res = 2.4145e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.132745', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.538874 GFlop/s', '# - bytes moved = 219.566 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.65404 GByte/s', '# - solve arithmetic intensity = 0.325792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 49.710s; elapsed, 22.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 53.230s; elapsed, 22.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.950s; elapsed, 22.302s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 24.540s; elapsed, 22.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.950s; elapsed, 22.302s, #calls: 1', 'TOTAL : CPU, 85.950s; elapsed, 22.302s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 53.230s; elapsed, 22.306s, #calls: 1', 'TOTAL : CPU, 53.230s; elapsed, 22.306s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 49.710s; elapsed, 22.318s, #calls: 1', 'TOTAL : CPU, 49.710s; elapsed, 22.318s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 89.460s; elapsed, 23.088s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.122s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.400s; elapsed, 38.877s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 24.540s; elapsed, 22.315s, #calls: 1', 'TOTAL : CPU, 153.620s; elapsed, 84.402s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 190.420s; elapsed, 24.606s', 'individual call time for EIGEN_LDLT: CPU, 43.400s; elapsed, 42.304s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.527541', '# - matching time = 0.0849051', '# - symmetrization time = 0.040694', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0853698', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 22.8869', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 224964, max = 7.03507e+10', '# - flop rate = 4.12768 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9961e-11\trel.res = 2.41402e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.117422', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.609194 GFlop/s', '# - bytes moved = 219.596 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87014 GByte/s', '# - solve arithmetic intensity = 0.325748 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 71.570s; elapsed, 23.826s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 114.980s; elapsed, 23.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 183.960s; elapsed, 23.810s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 29.120s; elapsed, 23.823s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 114.980s; elapsed, 23.815s, #calls: 1', 'TOTAL : CPU, 114.980s; elapsed, 23.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 183.960s; elapsed, 23.810s, #calls: 1', 'TOTAL : CPU, 183.960s; elapsed, 23.810s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 71.570s; elapsed, 23.826s, #calls: 1', 'TOTAL : CPU, 71.570s; elapsed, 23.826s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 190.420s; elapsed, 24.606s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 43.400s; elapsed, 42.304s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 29.120s; elapsed, 23.823s, #calls: 1', 'TOTAL : CPU, 263.250s; elapsed, 90.846s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 43.310s; elapsed, 43.400s', 'individual call time for EIGEN_LDLT: CPU, 42.900s; elapsed, 42.920s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.143s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# - nd time = 0.488085', '# - matching time = 0.0757408', '# - symmetrization time = 0.027653', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0586259', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 21.653', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 4.39562 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01808e-10\trel.res = 2.45864e-15\tbw.error = 4.93262e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.082864', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 0.982224 GFlop/s', '# - bytes moved = 212.161 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56035 GByte/s', '# - solve arithmetic intensity = 0.38363 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 22.420s; elapsed, 22.447s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.420s; elapsed, 22.447s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 22.380s; elapsed, 22.448s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 22.380s; elapsed, 22.437s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 22.350s; elapsed, 22.443s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.410s; elapsed, 22.444s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.390s; elapsed, 22.437s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.440s; elapsed, 22.450s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 22.350s; elapsed, 22.443s, #calls: 1', 'TOTAL : CPU, 22.350s; elapsed, 22.443s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.410s; elapsed, 22.444s, #calls: 1', 'TOTAL : CPU, 22.410s; elapsed, 22.444s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 22.380s; elapsed, 22.448s, #calls: 1', 'TOTAL : CPU, 22.380s; elapsed, 22.448s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.390s; elapsed, 22.437s, #calls: 1', 'TOTAL : CPU, 22.390s; elapsed, 22.437s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 22.420s; elapsed, 22.447s, #calls: 1', 'TOTAL : CPU, 22.420s; elapsed, 22.447s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.420s; elapsed, 22.447s, #calls: 1', 'TOTAL : CPU, 22.420s; elapsed, 22.447s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 22.380s; elapsed, 22.437s, #calls: 1', 'TOTAL : CPU, 22.380s; elapsed, 22.437s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 43.310s; elapsed, 43.400s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.143s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.900s; elapsed, 42.920s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.440s; elapsed, 22.450s, #calls: 1', 'TOTAL : CPU, 108.790s; elapsed, 108.913s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 50.520s; elapsed, 26.059s', 'individual call time for EIGEN_LDLT: CPU, 42.410s; elapsed, 42.261s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.500234', '# - matching time = 0.084003', '# - symmetrization time = 0.028939', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0678899', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 21.0922', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 4.51248 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01772e-10\trel.res = 2.45777e-15\tbw.error = 4.93262e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.081881', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 0.994016 GFlop/s', '# - bytes moved = 212.169 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.59118 GByte/s', '# - solve arithmetic intensity = 0.383615 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 26.520s; elapsed, 21.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.660s; elapsed, 21.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 29.790s; elapsed, 21.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 33.970s; elapsed, 21.913s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 42.350s; elapsed, 21.919s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.320s; elapsed, 21.919s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 24.010s; elapsed, 21.913s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.620s; elapsed, 21.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 26.520s; elapsed, 21.924s, #calls: 1', 'TOTAL : CPU, 26.520s; elapsed, 21.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 33.970s; elapsed, 21.913s, #calls: 1', 'TOTAL : CPU, 33.970s; elapsed, 21.913s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 24.010s; elapsed, 21.913s, #calls: 1', 'TOTAL : CPU, 24.010s; elapsed, 21.913s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 42.350s; elapsed, 21.919s, #calls: 1', 'TOTAL : CPU, 42.350s; elapsed, 21.919s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.320s; elapsed, 21.919s, #calls: 1', 'TOTAL : CPU, 24.320s; elapsed, 21.919s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.660s; elapsed, 21.924s, #calls: 1', 'TOTAL : CPU, 24.660s; elapsed, 21.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 29.790s; elapsed, 21.924s, #calls: 1', 'TOTAL : CPU, 29.790s; elapsed, 21.924s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 50.520s; elapsed, 26.059s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.410s; elapsed, 42.261s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.620s; elapsed, 21.926s, #calls: 1', 'TOTAL : CPU, 115.720s; elapsed, 90.366s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 110.570s; elapsed, 28.271s', 'individual call time for EIGEN_LDLT: CPU, 42.090s; elapsed, 41.753s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.106s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.521432', '# - matching time = 0.0837748', '# - symmetrization time = 0.041924', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0815721', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 23.6406', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 4.02606 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01768e-10\trel.res = 2.45766e-15\tbw.error = 4.92775e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0964129', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 0.844193 GFlop/s', '# - bytes moved = 212.179 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.20073 GByte/s', '# - solve arithmetic intensity = 0.383597 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 38.270s; elapsed, 24.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 32.080s; elapsed, 24.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 62.330s; elapsed, 24.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 50.220s; elapsed, 24.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 93.800s; elapsed, 24.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 31.250s; elapsed, 24.530s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 30.540s; elapsed, 24.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 26.700s; elapsed, 24.537s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 62.330s; elapsed, 24.523s, #calls: 1', 'TOTAL : CPU, 62.330s; elapsed, 24.523s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 38.270s; elapsed, 24.535s, #calls: 1', 'TOTAL : CPU, 38.270s; elapsed, 24.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 93.800s; elapsed, 24.529s, #calls: 1', 'TOTAL : CPU, 93.800s; elapsed, 24.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 31.250s; elapsed, 24.530s, #calls: 1', 'TOTAL : CPU, 31.250s; elapsed, 24.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 32.080s; elapsed, 24.535s, #calls: 1', 'TOTAL : CPU, 32.080s; elapsed, 24.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 50.220s; elapsed, 24.532s, #calls: 1', 'TOTAL : CPU, 50.220s; elapsed, 24.532s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 30.540s; elapsed, 24.523s, #calls: 1', 'TOTAL : CPU, 30.540s; elapsed, 24.523s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 110.570s; elapsed, 28.271s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.106s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.090s; elapsed, 41.753s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 26.700s; elapsed, 24.537s, #calls: 1', 'TOTAL : CPU, 179.570s; elapsed, 94.667s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 51.050s; elapsed, 51.150s', 'individual call time for EIGEN_LDLT: CPU, 47.770s; elapsed, 47.794s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.159s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,191', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.57309', '# - matching time = 0.082206', '# - symmetrization time = 0.027606', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,191', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.075449', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.38 MB', '# - factor time = 22.3297', '# - factor nonzeros = 31,297,498', '# - factor memory = 250.38 MB', '# - total flops = 9.44704e+10, min = 223014, max = 7.0108e+10', '# - flop rate = 4.2307 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.62837e+10', '# --------------------------------------------', '# total = 9.62837e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.88967e-11\trel.res = 1.66383e-15\tbw.error = 3.71324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0890191', '# - total flops = 1.00397e+08, min = 114336, max = 5.02255e+07', '# - flop rate = 1.12782 GFlop/s', '# - bytes moved = 204.061 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.29233 GByte/s', '# - solve arithmetic intensity = 0.491997 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 23.190s; elapsed, 23.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 23.200s; elapsed, 23.244s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 23.170s; elapsed, 23.244s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 23.180s; elapsed, 23.246s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 23.210s; elapsed, 23.246s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.140s; elapsed, 23.243s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 23.170s; elapsed, 23.244s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.160s; elapsed, 23.243s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 23.170s; elapsed, 23.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 23.180s; elapsed, 23.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 23.220s; elapsed, 23.245s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 23.210s; elapsed, 23.246s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 23.210s; elapsed, 23.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.210s; elapsed, 23.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 23.210s; elapsed, 23.243s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 23.250s; elapsed, 23.249s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 23.170s; elapsed, 23.244s, #calls: 1', 'TOTAL : CPU, 23.170s; elapsed, 23.244s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 23.190s; elapsed, 23.242s, #calls: 1', 'TOTAL : CPU, 23.190s; elapsed, 23.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.160s; elapsed, 23.243s, #calls: 1', 'TOTAL : CPU, 23.160s; elapsed, 23.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.140s; elapsed, 23.243s, #calls: 1', 'TOTAL : CPU, 23.140s; elapsed, 23.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 23.180s; elapsed, 23.246s, #calls: 1', 'TOTAL : CPU, 23.180s; elapsed, 23.246s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 23.210s; elapsed, 23.246s, #calls: 1', 'TOTAL : CPU, 23.210s; elapsed, 23.246s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 23.210s; elapsed, 23.242s, #calls: 1', 'TOTAL : CPU, 23.210s; elapsed, 23.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 23.180s; elapsed, 23.234s, #calls: 1', 'TOTAL : CPU, 23.180s; elapsed, 23.234s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 23.170s; elapsed, 23.244s, #calls: 1', 'TOTAL : CPU, 23.170s; elapsed, 23.244s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 23.210s; elapsed, 23.243s, #calls: 1', 'TOTAL : CPU, 23.210s; elapsed, 23.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.210s; elapsed, 23.242s, #calls: 1', 'TOTAL : CPU, 23.210s; elapsed, 23.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 23.220s; elapsed, 23.245s, #calls: 1', 'TOTAL : CPU, 23.220s; elapsed, 23.245s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 23.200s; elapsed, 23.244s, #calls: 1', 'TOTAL : CPU, 23.200s; elapsed, 23.244s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 23.170s; elapsed, 23.234s, #calls: 1', 'TOTAL : CPU, 23.170s; elapsed, 23.234s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 23.210s; elapsed, 23.246s, #calls: 1', 'TOTAL : CPU, 23.210s; elapsed, 23.246s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 51.050s; elapsed, 51.150s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.159s, #calls: 1', 'time for EIGEN_LDLT: CPU, 47.770s; elapsed, 47.794s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 23.250s; elapsed, 23.249s, #calls: 1', 'TOTAL : CPU, 122.220s; elapsed, 122.352s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 60.050s; elapsed, 31.120s', 'individual call time for EIGEN_LDLT: CPU, 43.510s; elapsed, 43.386s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,191', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.604434', '# - matching time = 0.081984', '# - symmetrization time = 0.027195', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,191', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0854731', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.38 MB', '# - factor time = 26.5573', '# - factor nonzeros = 31,297,498', '# - factor memory = 250.38 MB', '# - total flops = 9.44704e+10, min = 223014, max = 7.0108e+10', '# - flop rate = 3.55724 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.62837e+10', '# --------------------------------------------', '# total = 9.62837e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.88907e-11\trel.res = 1.66368e-15\tbw.error = 3.71324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.087384', '# - total flops = 1.00397e+08, min = 114336, max = 5.02255e+07', '# - flop rate = 1.14892 GFlop/s', '# - bytes moved = 204.065 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33527 GByte/s', '# - solve arithmetic intensity = 0.491988 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 29.910s; elapsed, 27.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 29.620s; elapsed, 27.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 30.080s; elapsed, 27.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 29.110s; elapsed, 27.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 35.290s; elapsed, 27.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 33.110s; elapsed, 27.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 29.290s; elapsed, 27.514sindividual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 37.090s; elapsed, 27.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 30.250s; elapsed, 27.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 30.320s; elapsed, 27.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 30.060s; elapsed, 27.514s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 33.260s; elapsed, 27.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 53.080s; elapsed, 27.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 29.870s; elapsed, 27.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 29.370s; elapsed, 27.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.210s; elapsed, 27.518s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 29.910s; elapsed, 27.513s, #calls: 1', 'TOTAL : CPU, 29.910s; elapsed, 27.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 29.290s; elapsed, 27.514s, #calls: 1', 'TOTAL : CPU, 29.290s; elapsed, 27.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 35.290s; elapsed, 27.512s, #calls: 1', 'TOTAL : CPU, 35.290s; elapsed, 27.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 30.060s; elapsed, 27.514s, #calls: 1', 'TOTAL : CPU, 30.060s; elapsed, 27.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 29.620s; elapsed, 27.515s, #calls: 1', 'TOTAL : CPU, 29.620s; elapsed, 27.515s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 37.090s; elapsed, 27.505s, #calls: 1', 'TOTAL : CPU, 37.090s; elapsed, 27.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 29.110s; elapsed, 27.505s, #calls: 1', 'TOTAL : CPU, 29.110s; elapsed, 27.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 30.080s; elapsed, 27.513s, #calls: 1', 'TOTAL : CPU, 30.080s; elapsed, 27.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 29.870s; elapsed, 27.512s, #calls: 1', 'TOTAL : CPU, 29.870s; elapsed, 27.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 30.320s; elapsed, 27.512s, #calls: 1', 'TOTAL : CPU, 30.320s; elapsed, 27.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 30.250s; elapsed, 27.512s, #calls: 1', 'TOTAL : CPU, 30.250s; elapsed, 27.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 53.080s; elapsed, 27.512s, #calls: 1', 'TOTAL : CPU, 53.080s; elapsed, 27.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 33.110s; elapsed, 27.513s, #calls: 1', 'TOTAL : CPU, 33.110s; elapsed, 27.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 29.370s; elapsed, 27.511s, #calls: 1', 'TOTAL : CPU, 29.370s; elapsed, 27.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 33.260s; elapsed, 27.512s, #calls: 1', 'TOTAL : CPU, 33.260s; elapsed, 27.512s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 60.050s; elapsed, 31.120s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 43.510s; elapsed, 43.386s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.210s; elapsed, 27.518s, #calls: 1', 'TOTAL : CPU, 131.930s; elapsed, 102.141s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 144.750s; elapsed, 144.883s', 'individual call time for EIGEN_LDLT: CPU, 292.640s; elapsed, 292.808s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.268s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 144.750s; elapsed, 144.883s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.268s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.640s; elapsed, 292.808s, #calls: 1', 'TOTAL : CPU, 437.650s; elapsed, 437.958s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 209.230s; elapsed, 106.380s', 'individual call time for EIGEN_LDLT: CPU, 292.950s; elapsed, 292.971s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.242s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 209.230s; elapsed, 106.380s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.242s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.950s; elapsed, 292.971s, #calls: 1', 'TOTAL : CPU, 502.540s; elapsed, 399.594s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 376.790s; elapsed, 95.569s', 'individual call time for EIGEN_LDLT: CPU, 292.920s; elapsed, 292.861s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.226s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 376.790s; elapsed, 95.569s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.226s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.920s; elapsed, 292.861s, #calls: 1', 'TOTAL : CPU, 670.200s; elapsed, 388.655s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 726.300s; elapsed, 93.282s', 'individual call time for EIGEN_LDLT: CPU, 322.660s; elapsed, 322.450s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.240s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 726.300s; elapsed, 93.282s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.240s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.660s; elapsed, 322.450s, #calls: 1', 'TOTAL : CPU, 1049.670s; elapsed, 415.972s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1886.140s; elapsed, 120.195s', 'individual call time for EIGEN_LDLT: CPU, 325.360s; elapsed, 322.904s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.070s; elapsed, 0.219s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1886.140s; elapsed, 120.195s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.070s; elapsed, 0.219s, #calls: 1', 'time for EIGEN_LDLT: CPU, 325.360s; elapsed, 322.904s, #calls: 1', 'TOTAL : CPU, 2212.570s; elapsed, 443.317s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 144.780s; elapsed, 144.891s', 'individual call time for EIGEN_LDLT: CPU, 292.700s; elapsed, 292.897s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.267s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.01134', '# - matching time = 0.125974', '# - symmetrization time = 0.130106', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.189003', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 224.994', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18828e+11, min = 20466, max = 7.18828e+11', '# - flop rate = 3.19488 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03767e-10\trel.res = 2.47992e-15\tbw.error = 8.30193e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.453427', '# - total flops = 2.36136e+08, min = 120284, max = 2.36016e+08', '# - flop rate = 0.52078 GFlop/s', '# - bytes moved = 1221.21 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.69329 GByte/s', '# - solve arithmetic intensity = 0.193362 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 226.610s; elapsed, 227.149s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 227.180s; elapsed, 227.165s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 226.610s; elapsed, 227.149s, #calls: 1', 'TOTAL : CPU, 226.610s; elapsed, 227.149s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 144.780s; elapsed, 144.891s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.267s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.700s; elapsed, 292.897s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 227.180s; elapsed, 227.165s, #calls: 1', 'TOTAL : CPU, 664.920s; elapsed, 665.221s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 214.360s; elapsed, 108.942s', 'individual call time for EIGEN_LDLT: CPU, 758.980s; elapsed, 759.273s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.560s; elapsed, 0.374s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using CPP -2', '2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.05939', '# - matching time = 0.141977', '# - symmetrization time = 0.131704', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.150567', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 98.8859', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18829e+11, min = 20466, max = 7.18829e+11', '# - flop rate = 7.26928 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05699e-10\trel.res = 2.52609e-15\tbw.error = 8.38845e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.285154', '# - total flops = 2.36212e+08, min = 120284, max = 2.36092e+08', '# - flop rate = 0.828369 GFlop/s', '# - bytes moved = 1223.28 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.28991 GByte/s', '# - solve arithmetic intensity = 0.193097 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 198.360s; elapsed, 100.828s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 101.950s; elapsed, 100.836s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 198.360s; elapsed, 100.828s, #calls: 1', 'TOTAL : CPU, 198.360s; elapsed, 100.828s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 214.360s; elapsed, 108.942s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.560s; elapsed, 0.374s, #calls: 1', 'time for EIGEN_LDLT: CPU, 758.980s; elapsed, 759.273s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 101.950s; elapsed, 100.836s, #calls: 1', 'TOTAL : CPU, 1075.850s; elapsed, 969.425s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 493.320s; elapsed, 127.419s', 'individual call time for EIGEN_LDLT: CPU, 322.900s; elapsed, 322.922s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.254s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.07369', '# - matching time = 0.131327', '# - symmetrization time = 0.132624', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.158763', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 106.726', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18829e+11, min = 20466, max = 7.18829e+11', '# - flop rate = 6.73526 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02652e-10\trel.res = 2.45327e-15\tbw.error = 8.38966e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.302491', '# - total flops = 2.36212e+08, min = 120284, max = 2.36092e+08', '# - flop rate = 0.780891 GFlop/s', '# - bytes moved = 1224.53 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.04815 GByte/s', '# - solve arithmetic intensity = 0.192901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 419.500s; elapsed, 108.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 112.190s; elapsed, 108.715s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 419.500s; elapsed, 108.702s, #calls: 1', 'TOTAL : CPU, 419.500s; elapsed, 108.702s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 493.320s; elapsed, 127.419s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.254s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.900s; elapsed, 322.922s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 112.190s; elapsed, 108.715s, #calls: 1', 'TOTAL : CPU, 928.920s; elapsed, 559.310s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 875.310s; elapsed, 112.047s', 'individual call time for EIGEN_LDLT: CPU, 323.580s; elapsed, 322.472s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.680s; elapsed, 0.230s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.0274', '# - matching time = 0.143564', '# - symmetrization time = 0.142047', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.16019', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 112.097', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18829e+11, min = 21139, max = 7.18829e+11', '# - flop rate = 6.41256 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02621e-10\trel.res = 2.45254e-15\tbw.error = 8.38966e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.455073', '# - total flops = 2.36212e+08, min = 120284, max = 2.36092e+08', '# - flop rate = 0.519065 GFlop/s', '# - bytes moved = 1224.6 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.691 GByte/s', '# - solve arithmetic intensity = 0.192889 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 887.340s; elapsed, 114.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 124.370s; elapsed, 114.324s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 887.340s; elapsed, 114.314s, #calls: 1', 'TOTAL : CPU, 887.340s; elapsed, 114.314s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 875.310s; elapsed, 112.047s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.680s; elapsed, 0.230s, #calls: 1', 'time for EIGEN_LDLT: CPU, 323.580s; elapsed, 322.472s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 124.370s; elapsed, 114.324s, #calls: 1', 'TOTAL : CPU, 1323.940s; elapsed, 549.074s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1853.790s; elapsed, 118.406s', 'individual call time for EIGEN_LDLT: CPU, 323.620s; elapsed, 323.079s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.950s; elapsed, 0.221s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', 'CPP -2', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,177', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.10518', '# - matching time = 0.161505', '# - symmetrization time = 0.159047', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,177', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.183789', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.344 MB', '# - factor time = 113.142', '# - factor nonzeros = 113,043,003', '# - factor memory = 904.344 MB', '# - total flops = 7.18833e+11, min = 21720, max = 7.18833e+11', '# - flop rate = 6.35334 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28746e+11', '# --------------------------------------------', '# total = 7.28746e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0254e-10\trel.res = 2.45058e-15\tbw.error = 8.38966e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.319274', '# - total flops = 2.36222e+08, min = 120284, max = 2.36102e+08', '# - flop rate = 0.739871 GFlop/s', '# - bytes moved = 1224.9 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.83651 GByte/s', '# - solve arithmetic intensity = 0.19285 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1809.300s; elapsed, 115.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 133.630s; elapsed, 115.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1809.300s; elapsed, 115.274s, #calls: 1', 'TOTAL : CPU, 1809.300s; elapsed, 115.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1853.790s; elapsed, 118.406s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.950s; elapsed, 0.221s, #calls: 1', 'time for EIGEN_LDLT: CPU, 323.620s; elapsed, 323.079s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 133.630s; elapsed, 115.281s, #calls: 1', 'TOTAL : CPU, 2311.990s; elapsed, 556.987s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 225.050s; elapsed, 225.619s', 'individual call time for EIGEN_LDLT: CPU, 326.280s; elapsed, 326.429s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.327s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.16659', '# - matching time = 0.133076', '# - symmetrization time = 0.0911331', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.151328', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 102.201', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4725e+11', '# - flop rate = 6.95796 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.12597e-11\trel.res = 2.181e-15\tbw.error = 7.73321e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.288385', '# - total flops = 3.02508e+08, min = 82374, max = 1.07552e+08', '# - flop rate = 1.04897 GFlop/s', '# - bytes moved = 501.232 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.73807 GByte/s', '# - solve arithmetic intensity = 0.603529 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 104.000s; elapsed, 104.162s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 104.120s; elapsed, 104.150s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 103.950s; elapsed, 104.139s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 104.150s; elapsed, 104.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 104.000s; elapsed, 104.162s, #calls: 1', 'TOTAL : CPU, 104.000s; elapsed, 104.162s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 104.120s; elapsed, 104.150s, #calls: 1', 'TOTAL : CPU, 104.120s; elapsed, 104.150s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 103.950s; elapsed, 104.139s, #calls: 1', 'TOTAL : CPU, 103.950s; elapsed, 104.139s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 225.050s; elapsed, 225.619s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.327s, #calls: 1', 'time for EIGEN_LDLT: CPU, 326.280s; elapsed, 326.429s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 104.150s; elapsed, 104.164s, #calls: 1', 'TOTAL : CPU, 655.810s; elapsed, 656.539s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 284.740s; elapsed, 147.310s', 'individual call time for EIGEN_LDLT: CPU, 332.600s; elapsed, 332.622s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.285s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.24997', '# - matching time = 0.140286', '# - symmetrization time = 0.0887129', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.136533', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 103.337', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4725e+11', '# - flop rate = 6.88142 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.10797e-11\trel.res = 2.1767e-15\tbw.error = 7.7354e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.355509', '# - total flops = 3.02534e+08, min = 82374, max = 1.07578e+08', '# - flop rate = 0.850988 GFlop/s', '# - bytes moved = 502.373 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.41311 GByte/s', '# - solve arithmetic intensity = 0.602209 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 179.200s; elapsed, 105.460s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 122.880s; elapsed, 105.460s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 204.220s; elapsed, 105.448s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 106.610s; elapsed, 105.462s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 122.880s; elapsed, 105.460s, #calls: 1', 'TOTAL : CPU, 122.880s; elapsed, 105.460s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 179.200s; elapsed, 105.460s, #calls: 1', 'TOTAL : CPU, 179.200s; elapsed, 105.460s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 204.220s; elapsed, 105.448s, #calls: 1', 'TOTAL : CPU, 204.220s; elapsed, 105.448s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 284.740s; elapsed, 147.310s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.285s, #calls: 1', 'time for EIGEN_LDLT: CPU, 332.600s; elapsed, 332.622s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 106.610s; elapsed, 105.462s, #calls: 1', 'TOTAL : CPU, 724.360s; elapsed, 585.679s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 443.450s; elapsed, 115.919s', 'individual call time for EIGEN_LDLT: CPU, 327.220s; elapsed, 327.201s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.258s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.24828', '# - matching time = 0.191711', '# - symmetrization time = 0.0979841', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.145002', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 96.7182', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4725e+11', '# - flop rate = 7.35236 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.10174e-11\trel.res = 2.17521e-15\tbw.error = 7.7354e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.421026', '# - total flops = 3.02534e+08, min = 82374, max = 1.07578e+08', '# - flop rate = 0.718563 GFlop/s', '# - bytes moved = 502.387 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.19324 GByte/s', '# - solve arithmetic intensity = 0.602193 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 246.670s; elapsed, 98.990s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 162.220s; elapsed, 98.981s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 373.720s; elapsed, 98.968s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 102.740s; elapsed, 98.992s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 373.720s; elapsed, 98.968s, #calls: 1', 'TOTAL : CPU, 373.720s; elapsed, 98.968s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 246.670s; elapsed, 98.990s, #calls: 1', 'TOTAL : CPU, 246.670s; elapsed, 98.990s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 162.220s; elapsed, 98.981s, #calls: 1', 'TOTAL : CPU, 162.220s; elapsed, 98.981s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 443.450s; elapsed, 115.919s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.258s, #calls: 1', 'time for EIGEN_LDLT: CPU, 327.220s; elapsed, 327.201s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 102.740s; elapsed, 98.992s, #calls: 1', 'TOTAL : CPU, 873.920s; elapsed, 542.370s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 838.800s; elapsed, 108.606s', 'individual call time for EIGEN_LDLT: CPU, 359.790s; elapsed, 359.424s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.720s; elapsed, 0.244s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 69', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.25453', '# - matching time = 0.170377', '# - symmetrization time = 0.124878', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.156058', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 96.0153', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 21139, max = 3.4725e+11', '# - flop rate = 7.40619 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.21369e+11', '# --------------------------------------------', '# total = 7.21369e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.09459e-11\trel.res = 2.1735e-15\tbw.error = 7.7354e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.432069', '# - total flops = 3.02534e+08, min = 82374, max = 1.07578e+08', '# - flop rate = 0.700198 GFlop/s', '# - bytes moved = 502.416 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.16281 GByte/s', '# - solve arithmetic intensity = 0.602158 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 482.950s; elapsed, 98.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 729.120s; elapsed, 98.295s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 238.790s; elapsed, 98.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 107.100s; elapsed, 98.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 729.120s; elapsed, 98.295s, #calls: 1', 'TOTAL : CPU, 729.120s; elapsed, 98.295s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 482.950s; elapsed, 98.317s, #calls: 1', 'TOTAL : CPU, 482.950s; elapsed, 98.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 238.790s; elapsed, 98.309s, #calls: 1', 'TOTAL : CPU, 238.790s; elapsed, 98.309s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 838.800s; elapsed, 108.606s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.720s; elapsed, 0.244s, #calls: 1', 'time for EIGEN_LDLT: CPU, 359.790s; elapsed, 359.424s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 107.100s; elapsed, 98.314s, #calls: 1', 'TOTAL : CPU, 1306.410s; elapsed, 566.588s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 158.790s; elapsed, 159.010s', 'individual call time for EIGEN_LDLT: CPU, 309.360s; elapsed, 309.564s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.302s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.23504', '# - matching time = 0.144766', '# - symmetrization time = 0.0674059', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.132333', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 84.0179', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.47059e+11', '# - flop rate = 8.46376 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.20596e+11', '# --------------------------------------------', '# total = 7.20596e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.04956e-11\trel.res = 1.68477e-15\tbw.error = 5.68015e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.229835', '# - total flops = 6.03194e+08, min = 63419, max = 1.00803e+08', '# - flop rate = 2.62447 GFlop/s', '# - bytes moved = 447.163 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94558 GByte/s', '# - solve arithmetic intensity = 1.34894 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 85.740s; elapsed, 85.938s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 85.880s; elapsed, 85.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.850s; elapsed, 85.925s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 85.690s; elapsed, 85.925s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 85.880s; elapsed, 85.930s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 85.890s; elapsed, 85.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 85.670s; elapsed, 85.918s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.920s; elapsed, 85.936s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 85.880s; elapsed, 85.930s, #calls: 1', 'TOTAL : CPU, 85.880s; elapsed, 85.930s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 85.690s; elapsed, 85.925s, #calls: 1', 'TOTAL : CPU, 85.690s; elapsed, 85.925s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 85.670s; elapsed, 85.918s, #calls: 1', 'TOTAL : CPU, 85.670s; elapsed, 85.918s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 85.880s; elapsed, 85.928s, #calls: 1', 'TOTAL : CPU, 85.880s; elapsed, 85.928s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 85.740s; elapsed, 85.938s, #calls: 1', 'TOTAL : CPU, 85.740s; elapsed, 85.938s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 85.890s; elapsed, 85.922s, #calls: 1', 'TOTAL : CPU, 85.890s; elapsed, 85.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.850s; elapsed, 85.925s, #calls: 1', 'TOTAL : CPU, 85.850s; elapsed, 85.925s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 158.790s; elapsed, 159.010s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.302s, #calls: 1', 'time for EIGEN_LDLT: CPU, 309.360s; elapsed, 309.564s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.920s; elapsed, 85.936s, #calls: 1', 'TOTAL : CPU, 554.370s; elapsed, 554.812s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 247.550s; elapsed, 128.281s', 'individual call time for EIGEN_LDLT: CPU, 329.360s; elapsed, 329.506s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.291s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.26009', '# - matching time = 0.139356', '# - symmetrization time = 0.0610461', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.125814', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 107.402', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.47059e+11', '# - flop rate = 6.62102 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.20596e+11', '# --------------------------------------------', '# total = 7.20596e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.02679e-11\trel.res = 1.67933e-15\tbw.error = 5.68236e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198102', '# - total flops = 6.03204e+08, min = 63419, max = 1.00803e+08', '# - flop rate = 3.04492 GFlop/s', '# - bytes moved = 447.576 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25932 GByte/s', '# - solve arithmetic intensity = 1.34771 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 145.030s; elapsed, 109.294s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 124.250s; elapsed, 109.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 129.510s; elapsed, 109.281s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 122.650s; elapsed, 109.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 210.250s; elapsed, 109.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 122.030s; elapsed, 109.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 173.320s; elapsed, 109.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 110.230s; elapsed, 109.293s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 122.650s; elapsed, 109.286s, #calls: 1', 'TOTAL : CPU, 122.650s; elapsed, 109.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 124.250s; elapsed, 109.283s, #calls: 1', 'TOTAL : CPU, 124.250s; elapsed, 109.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 122.030s; elapsed, 109.276s, #calls: 1', 'TOTAL : CPU, 122.030s; elapsed, 109.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 173.320s; elapsed, 109.276s, #calls: 1', 'TOTAL : CPU, 173.320s; elapsed, 109.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 129.510s; elapsed, 109.281s, #calls: 1', 'TOTAL : CPU, 129.510s; elapsed, 109.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 210.250s; elapsed, 109.282s, #calls: 1', 'TOTAL : CPU, 210.250s; elapsed, 109.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 145.030s; elapsed, 109.294s, #calls: 1', 'TOTAL : CPU, 145.030s; elapsed, 109.294s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 247.550s; elapsed, 128.281s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.291s, #calls: 1', 'time for EIGEN_LDLT: CPU, 329.360s; elapsed, 329.506s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 110.230s; elapsed, 109.293s, #calls: 1', 'TOTAL : CPU, 687.570s; elapsed, 567.371s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 470.230s; elapsed, 122.928s', 'individual call time for EIGEN_LDLT: CPU, 326.180s; elapsed, 326.117s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.247s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.16167', '# - matching time = 0.158653', '# - symmetrization time = 0.070461', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.148759', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 103.746', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.47059e+11', '# - flop rate = 6.85433 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.20596e+11', '# --------------------------------------------', '# total = 7.20596e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.01929e-11\trel.res = 1.67753e-15\tbw.error = 5.68236e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.208501', '# - total flops = 6.03204e+08, min = 63419, max = 1.00803e+08', '# - flop rate = 2.89305 GFlop/s', '# - bytes moved = 447.589 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.1467 GByte/s', '# - solve arithmetic intensity = 1.34767 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 240.760s; elapsed, 105.626s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 136.590s; elapsed, 105.613s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 132.450s; elapsed, 105.615s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 393.380s; elapsed, 105.612s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 167.960s; elapsed, 105.614s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 130.930s; elapsed, 105.608s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 268.990s; elapsed, 105.607s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 108.660s; elapsed, 105.622s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 240.760s; elapsed, 105.626s, #calls: 1', 'TOTAL : CPU, 240.760s; elapsed, 105.626s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 130.930s; elapsed, 105.608s, #calls: 1', 'TOTAL : CPU, 130.930s; elapsed, 105.608s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 136.590s; elapsed, 105.613s, #calls: 1', 'TOTAL : CPU, 136.590s; elapsed, 105.613s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 393.380s; elapsed, 105.612s, #calls: 1', 'TOTAL : CPU, 393.380s; elapsed, 105.612s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 132.450s; elapsed, 105.615s, #calls: 1', 'TOTAL : CPU, 132.450s; elapsed, 105.615s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 268.990s; elapsed, 105.607s, #calls: 1', 'TOTAL : CPU, 268.990s; elapsed, 105.607s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 167.960s; elapsed, 105.614s, #calls: 1', 'TOTAL : CPU, 167.960s; elapsed, 105.614s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 470.230s; elapsed, 122.928s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.247s, #calls: 1', 'time for EIGEN_LDLT: CPU, 326.180s; elapsed, 326.117s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 108.660s; elapsed, 105.622s, #calls: 1', 'TOTAL : CPU, 905.570s; elapsed, 554.915s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 245.660s; elapsed, 246.368s', 'individual call time for EIGEN_LDLT: CPU, 347.660s; elapsed, 347.875s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.359s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.32139', '# - matching time = 0.152482', '# - symmetrization time = 0.0548592', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.136478', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 125.508', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4671e+11', '# - flop rate = 5.66583 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.19036e+11', '# --------------------------------------------', '# total = 7.19036e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.52689e-11\trel.res = 1.55985e-15\tbw.error = 6.33399e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.137449', '# - total flops = 1.0636e+09, min = 53944, max = 1.00793e+08', '# - flop rate = 7.73812 GFlop/s', '# - bytes moved = 420.668 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.06054 GByte/s', '# - solve arithmetic intensity = 2.52835 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 126.920s; elapsed, 127.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 127.320s; elapsed, 127.403s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 127.190s; elapsed, 127.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 127.260s; elapsed, 127.409s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 127.170s; elapsed, 127.410s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 127.290s; elapsed, 127.400s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 127.280s; elapsed, 127.400s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 127.290s; elapsed, 127.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 127.280s; elapsed, 127.403s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 127.340s; elapsed, 127.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 127.280s; elapsed, 127.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 127.340s; elapsed, 127.400s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.270s; elapsed, 127.403s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 127.320s; elapsed, 127.403s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 127.070s; elapsed, 127.401s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 127.360s; elapsed, 127.411s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 126.920s; elapsed, 127.405s, #calls: 1', 'TOTAL : CPU, 126.920s; elapsed, 127.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 127.320s; elapsed, 127.403s, #calls: 1', 'TOTAL : CPU, 127.320s; elapsed, 127.403s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 127.340s; elapsed, 127.400s, #calls: 1', 'TOTAL : CPU, 127.340s; elapsed, 127.400s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 127.320s; elapsed, 127.403s, #calls: 1', 'TOTAL : CPU, 127.320s; elapsed, 127.403s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 127.280s; elapsed, 127.404s, #calls: 1', 'TOTAL : CPU, 127.280s; elapsed, 127.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 127.280s; elapsed, 127.400s, #calls: 1', 'TOTAL : CPU, 127.280s; elapsed, 127.400s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 127.280s; elapsed, 127.403s, #calls: 1', 'TOTAL : CPU, 127.280s; elapsed, 127.403s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 127.260s; elapsed, 127.409s, #calls: 1', 'TOTAL : CPU, 127.260s; elapsed, 127.409s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 127.290s; elapsed, 127.404s, #calls: 1', 'TOTAL : CPU, 127.290s; elapsed, 127.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 127.190s; elapsed, 127.405s, #calls: 1', 'TOTAL : CPU, 127.190s; elapsed, 127.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 127.290s; elapsed, 127.400s, #calls: 1', 'TOTAL : CPU, 127.290s; elapsed, 127.400s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 127.340s; elapsed, 127.404s, #calls: 1', 'TOTAL : CPU, 127.340s; elapsed, 127.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 127.170s; elapsed, 127.410s, #calls: 1', 'TOTAL : CPU, 127.170s; elapsed, 127.410s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 127.070s; elapsed, 127.401s, #calls: 1', 'TOTAL : CPU, 127.070s; elapsed, 127.401s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.270s; elapsed, 127.403s, #calls: 1', 'TOTAL : CPU, 127.270s; elapsed, 127.403s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 245.660s; elapsed, 246.368s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.359s, #calls: 1', 'time for EIGEN_LDLT: CPU, 347.660s; elapsed, 347.875s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 127.360s; elapsed, 127.411s, #calls: 1', 'TOTAL : CPU, 721.040s; elapsed, 722.013s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 322.260s; elapsed, 166.928s', 'individual call time for EIGEN_LDLT: CPU, 336.120s; elapsed, 336.258s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.341s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 1,206,277', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,617', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.33817', '# - matching time = 0.153346', '# - symmetrization time = 0.055182', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,617', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.162958', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 893.148 MB', '# - factor time = 126.953', '# - factor nonzeros = 111,643,525', '# - factor memory = 893.148 MB', '# - total flops = 7.11108e+11, min = 20466, max = 3.4671e+11', '# - flop rate = 5.60136 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.19036e+11', '# --------------------------------------------', '# total = 7.19036e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.50696e-11\trel.res = 1.55509e-15\tbw.error = 6.33399e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.136756', '# - total flops = 1.0636e+09, min = 53944, max = 1.00793e+08', '# - flop rate = 7.77732 GFlop/s', '# - bytes moved = 420.681 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.07614 GByte/s', '# - solve arithmetic intensity = 2.52827 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 152.860s; elapsed, 128.899s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 134.700s; elapsed, 128.889s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 169.670s; elapsed, 128.889s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 138.250s; elapsed, 128.890s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 137.750s; elapsed, 128.888s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 136.850s; elapsed, 128.888s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 168.320s; elapsed, 128.896s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 135.500s; elapsed, 128.898s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 149.850s; elapsed, 128.893s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 242.090s; elapsed, 128.893s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 136.960s; elapsed, 128.894s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 136.690s; elapsed, 128.892s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 136.140s; elapsed, 128.893s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 135.470s; elapsed, 128.893s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 135.550s; elapsed, 128.895s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 129.780s; elapsed, 128.901s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 137.750s; elapsed, 128.888s, #calls: 1', 'TOTAL : CPU, 137.750s; elapsed, 128.888s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 138.250s; elapsed, 128.890s, #calls: 1', 'TOTAL : CPU, 138.250s; elapsed, 128.890s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 152.860s; elapsed, 128.899s, #calls: 1', 'TOTAL : CPU, 152.860s; elapsed, 128.899s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 135.550s; elapsed, 128.895s, #calls: 1', 'TOTAL : CPU, 135.550s; elapsed, 128.895s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 242.090s; elapsed, 128.893s, #calls: 1', 'TOTAL : CPU, 242.090s; elapsed, 128.893s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 134.700s; elapsed, 128.889s, #calls: 1', 'TOTAL : CPU, 134.700s; elapsed, 128.889s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 136.690s; elapsed, 128.892s, #calls: 1', 'TOTAL : CPU, 136.690s; elapsed, 128.892s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 168.320s; elapsed, 128.896s, #calls: 1', 'TOTAL : CPU, 168.320s; elapsed, 128.896s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 149.850s; elapsed, 128.893s, #calls: 1', 'TOTAL : CPU, 149.850s; elapsed, 128.893s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 135.470s; elapsed, 128.893s, #calls: 1', 'TOTAL : CPU, 135.470s; elapsed, 128.893s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 136.850s; elapsed, 128.888s, #calls: 1', 'TOTAL : CPU, 136.850s; elapsed, 128.888s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 135.500s; elapsed, 128.898s, #calls: 1', 'TOTAL : CPU, 135.500s; elapsed, 128.898s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 136.140s; elapsed, 128.893s, #calls: 1', 'TOTAL : CPU, 136.140s; elapsed, 128.893s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 169.670s; elapsed, 128.889s, #calls: 1', 'TOTAL : CPU, 169.670s; elapsed, 128.889s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 136.960s; elapsed, 128.894s, #calls: 1', 'TOTAL : CPU, 136.960s; elapsed, 128.894s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 322.260s; elapsed, 166.928s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.341s, #calls: 1', 'time for EIGEN_LDLT: CPU, 336.120s; elapsed, 336.258s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 129.780s; elapsed, 128.901s, #calls: 1', 'TOTAL : CPU, 788.640s; elapsed, 632.428s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.170s; elapsed, 146.296s', 'individual call time for EIGEN_LDLT: CPU, 292.610s; elapsed, 292.828s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.170s; elapsed, 146.296s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.274s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.610s; elapsed, 292.828s, #calls: 1', 'TOTAL : CPU, 439.060s; elapsed, 439.397s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 210.350s; elapsed, 107.075s', 'individual call time for EIGEN_LDLT: CPU, 292.950s; elapsed, 292.942s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.272s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 210.350s; elapsed, 107.075s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.272s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.950s; elapsed, 292.942s, #calls: 1', 'TOTAL : CPU, 503.700s; elapsed, 400.289s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 433.490s; elapsed, 111.928s', 'individual call time for EIGEN_LDLT: CPU, 294.040s; elapsed, 293.721s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.238s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 433.490s; elapsed, 111.928s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.238s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.040s; elapsed, 293.721s, #calls: 1', 'TOTAL : CPU, 728.050s; elapsed, 405.887s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 846.830s; elapsed, 109.180s', 'individual call time for EIGEN_LDLT: CPU, 411.120s; elapsed, 410.896s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.750s; elapsed, 0.310s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 846.830s; elapsed, 109.180s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.750s; elapsed, 0.310s, #calls: 1', 'time for EIGEN_LDLT: CPU, 411.120s; elapsed, 410.896s, #calls: 1', 'TOTAL : CPU, 1258.700s; elapsed, 520.386s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 40321.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1861.150s; elapsed, 118.430s', 'individual call time for EIGEN_LDLT: CPU, 294.540s; elapsed, 293.659s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.150s; elapsed, 0.215s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1861.150s; elapsed, 118.430s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.150s; elapsed, 0.215s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.540s; elapsed, 293.659s, #calls: 1', 'TOTAL : CPU, 2156.840s; elapsed, 412.304s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 202.310s; elapsed, 202.833s', 'individual call time for EIGEN_LDLT: CPU, 322.110s; elapsed, 322.301s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.324s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.2621', '# - matching time = 0.136667', '# - symmetrization time = 0.0852621', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.160482', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 142.465', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17167e+11, min = 222721, max = 7.17167e+11', '# - flop rate = 5.03397 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15237e-10\trel.res = 2.75403e-15\tbw.error = 7.73582e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.262851', '# - total flops = 2.35974e+08, min = 302329, max = 2.35671e+08', '# - flop rate = 0.897747 GFlop/s', '# - bytes moved = 1221.18 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.64589 GByte/s', '# - solve arithmetic intensity = 0.193235 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 144.420s; elapsed, 144.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 144.530s; elapsed, 144.537s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 144.420s; elapsed, 144.561s, #calls: 1', 'TOTAL : CPU, 144.420s; elapsed, 144.561s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 202.310s; elapsed, 202.833s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.324s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.110s; elapsed, 322.301s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 144.530s; elapsed, 144.537s, #calls: 1', 'TOTAL : CPU, 669.270s; elapsed, 669.996s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 211.620s; elapsed, 107.802s', 'individual call time for EIGEN_LDLT: CPU, 655.960s; elapsed, 656.087s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.640s; elapsed, 0.398s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.20423', '# - matching time = 0.144975', '# - symmetrization time = 0.0894849', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.170379', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 101.947', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17168e+11, min = 222721, max = 7.17168e+11', '# - flop rate = 7.03468 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.19346e-10\trel.res = 2.85224e-15\tbw.error = 7.68484e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.397157', '# - total flops = 2.36048e+08, min = 302329, max = 2.35746e+08', '# - flop rate = 0.594344 GFlop/s', '# - bytes moved = 1223.21 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.07991 GByte/s', '# - solve arithmetic intensity = 0.192974 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 203.770s; elapsed, 104.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 105.590s; elapsed, 104.230s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 203.770s; elapsed, 104.258s, #calls: 1', 'TOTAL : CPU, 203.770s; elapsed, 104.258s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 211.620s; elapsed, 107.802s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.640s; elapsed, 0.398s, #calls: 1', 'time for EIGEN_LDLT: CPU, 655.960s; elapsed, 656.087s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 105.590s; elapsed, 104.230s, #calls: 1', 'TOTAL : CPU, 973.810s; elapsed, 868.517s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 504.800s; elapsed, 130.481s', 'individual call time for EIGEN_LDLT: CPU, 322.510s; elapsed, 322.480s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.280s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.23248', '# - matching time = 0.1444', '# - symmetrization time = 0.087466', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.146381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 111.1', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17168e+11, min = 222721, max = 7.17168e+11', '# - flop rate = 6.45515 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18787e-10\trel.res = 2.83887e-15\tbw.error = 7.68484e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.295109', '# - total flops = 2.36048e+08, min = 302329, max = 2.35746e+08', '# - flop rate = 0.799867 GFlop/s', '# - bytes moved = 1224.43 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.14907 GByte/s', '# - solve arithmetic intensity = 0.192782 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 435.640s; elapsed, 113.289s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 116.940s; elapsed, 113.263s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 435.640s; elapsed, 113.289s, #calls: 1', 'TOTAL : CPU, 435.640s; elapsed, 113.289s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 504.800s; elapsed, 130.481s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.280s, #calls: 1', 'time for EIGEN_LDLT: CPU, 322.510s; elapsed, 322.480s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 116.940s; elapsed, 113.263s, #calls: 1', 'TOTAL : CPU, 944.820s; elapsed, 566.503s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 776.240s; elapsed, 99.960s', 'individual call time for EIGEN_LDLT: CPU, 501.750s; elapsed, 501.797s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.720s; elapsed, 0.250s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.2441', '# - matching time = 0.144549', '# - symmetrization time = 0.101758', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.209405', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 115.601', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17168e+11, min = 224411, max = 7.17168e+11', '# - flop rate = 6.20382 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18709e-10\trel.res = 2.837e-15\tbw.error = 7.69021e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.382594', '# - total flops = 2.36048e+08, min = 302329, max = 2.35746e+08', '# - flop rate = 0.616967 GFlop/s', '# - bytes moved = 1224.5 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.20052 GByte/s', '# - solve arithmetic intensity = 0.192771 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 921.860s; elapsed, 118.009s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 127.950s; elapsed, 117.984s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 921.860s; elapsed, 118.009s, #calls: 1', 'TOTAL : CPU, 921.860s; elapsed, 118.009s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 776.240s; elapsed, 99.960s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.720s; elapsed, 0.250s, #calls: 1', 'time for EIGEN_LDLT: CPU, 501.750s; elapsed, 501.797s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 127.950s; elapsed, 117.984s, #calls: 1', 'TOTAL : CPU, 1406.660s; elapsed, 719.991s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 20160.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1871.730s; elapsed, 119.367s', 'individual call time for EIGEN_LDLT: CPU, 373.040s; elapsed, 372.452s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.190s; elapsed, 0.227s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,259', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.32153', '# - matching time = 0.171016', '# - symmetrization time = 0.106983', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,259', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.182426', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.35 MB', '# - factor time = 71.9561', '# - factor nonzeros = 112,918,709', '# - factor memory = 903.35 MB', '# - total flops = 7.17172e+11, min = 226606, max = 7.17172e+11', '# - flop rate = 9.96679 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.271e+11', '# --------------------------------------------', '# total = 7.271e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18656e-10\trel.res = 2.83575e-15\tbw.error = 7.69557e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.551599', '# - total flops = 2.36057e+08, min = 302329, max = 2.35755e+08', '# - flop rate = 0.427951 GFlop/s', '# - bytes moved = 1224.79 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.22044 GByte/s', '# - solve arithmetic intensity = 0.192732 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1154.550s; elapsed, 74.637s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 96.500s; elapsed, 74.608s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1154.550s; elapsed, 74.637s, #calls: 1', 'TOTAL : CPU, 1154.550s; elapsed, 74.637s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1871.730s; elapsed, 119.367s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.190s; elapsed, 0.227s, #calls: 1', 'time for EIGEN_LDLT: CPU, 373.040s; elapsed, 372.452s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 96.500s; elapsed, 74.608s, #calls: 1', 'TOTAL : CPU, 2342.460s; elapsed, 566.653s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 152.290s; elapsed, 152.432s', 'individual call time for EIGEN_LDLT: CPU, 302.020s; elapsed, 302.219s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.286s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.24281', '# - matching time = 0.133822', '# - symmetrization time = 0.101481', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.111487', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 120.6', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 223014, max = 5.53072e+11', '# - flop rate = 5.97778 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.60424e-11\trel.res = 2.29531e-15\tbw.error = 7.82955e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.20445', '# - total flops = 2.51898e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 1.23207 GFlop/s', '# - bytes moved = 462.411 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26173 GByte/s', '# - solve arithmetic intensity = 0.544749 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 122.330s; elapsed, 122.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 122.440s; elapsed, 122.517s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 122.150s; elapsed, 122.508s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 122.520s; elapsed, 122.526s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 122.330s; elapsed, 122.540s, #calls: 1', 'TOTAL : CPU, 122.330s; elapsed, 122.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 122.150s; elapsed, 122.508s, #calls: 1', 'TOTAL : CPU, 122.150s; elapsed, 122.508s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 122.440s; elapsed, 122.517s, #calls: 1', 'TOTAL : CPU, 122.440s; elapsed, 122.517s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 152.290s; elapsed, 152.432s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.286s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.020s; elapsed, 302.219s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 122.520s; elapsed, 122.526s, #calls: 1', 'TOTAL : CPU, 577.120s; elapsed, 577.463s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 278.830s; elapsed, 144.077s', 'individual call time for EIGEN_LDLT: CPU, 564.830s; elapsed, 565.425s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.302s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.2854', '# - matching time = 0.141896', '# - symmetrization time = 0.0974059', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.125622', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 74.2078', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 223014, max = 5.53072e+11', '# - flop rate = 9.7149 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.66947e-11\trel.res = 2.31089e-15\tbw.error = 7.81571e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.18555', '# - total flops = 2.51909e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 1.35764 GFlop/s', '# - bytes moved = 462.869 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.49458 GByte/s', '# - solve arithmetic intensity = 0.544234 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 125.310s; elapsed, 76.200s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 138.170s; elapsed, 76.178s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 148.960s; elapsed, 76.168s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 77.130s; elapsed, 76.183s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 138.170s; elapsed, 76.178s, #calls: 1', 'TOTAL : CPU, 138.170s; elapsed, 76.178s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 148.960s; elapsed, 76.168s, #calls: 1', 'TOTAL : CPU, 148.960s; elapsed, 76.168s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 125.310s; elapsed, 76.200s, #calls: 1', 'TOTAL : CPU, 125.310s; elapsed, 76.200s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 278.830s; elapsed, 144.077s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.302s, #calls: 1', 'time for EIGEN_LDLT: CPU, 564.830s; elapsed, 565.425s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 77.130s; elapsed, 76.183s, #calls: 1', 'TOTAL : CPU, 921.240s; elapsed, 785.986s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 480.620s; elapsed, 125.417s', 'individual call time for EIGEN_LDLT: CPU, 324.320s; elapsed, 324.338s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.300s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.2437', '# - matching time = 0.143818', '# - symmetrization time = 0.10511', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.147142', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 97.92', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 223014, max = 5.53072e+11', '# - flop rate = 7.36235 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.58014e-11\trel.res = 2.28955e-15\tbw.error = 7.811e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.329068', '# - total flops = 2.51909e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 0.765523 GFlop/s', '# - bytes moved = 462.883 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.40665 GByte/s', '# - solve arithmetic intensity = 0.544218 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 225.100s; elapsed, 100.051s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 235.980s; elapsed, 100.028s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 387.280s; elapsed, 100.019s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 103.460s; elapsed, 100.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 225.100s; elapsed, 100.051s, #calls: 1', 'TOTAL : CPU, 225.100s; elapsed, 100.051s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 235.980s; elapsed, 100.028s, #calls: 1', 'TOTAL : CPU, 235.980s; elapsed, 100.028s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 387.280s; elapsed, 100.019s, #calls: 1', 'TOTAL : CPU, 387.280s; elapsed, 100.019s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 480.620s; elapsed, 125.417s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.300s, #calls: 1', 'time for EIGEN_LDLT: CPU, 324.320s; elapsed, 324.338s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 103.460s; elapsed, 100.035s, #calls: 1', 'TOTAL : CPU, 909.000s; elapsed, 550.090s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 10080.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 856.740s; elapsed, 110.154s', 'individual call time for EIGEN_LDLT: CPU, 442.120s; elapsed, 441.943s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.720s; elapsed, 0.240s', 'CPP -2', '# Initializing STRUMPACK', '# using 8CPP -2', 'CPP -2', 'CPP -2', ' OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,253', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.26725', '# - matching time = 0.173441', '# - symmetrization time = 0.117944', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,253', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.137071', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 906.042 MB', '# - factor time = 134.645', '# - factor nonzeros = 113,255,191', '# - factor memory = 906.042 MB', '# - total flops = 7.20921e+11, min = 224964, max = 5.53072e+11', '# - flop rate = 5.35422 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.305e+11', '# --------------------------------------------', '# total = 7.305e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.57902e-11\trel.res = 2.28928e-15\tbw.error = 7.811e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.336492', '# - total flops = 2.51909e+08, min = 252013, max = 1.89316e+08', '# - flop rate = 0.748633 GFlop/s', '# - bytes moved = 462.912 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.3757 GByte/s', '# - solve arithmetic intensity = 0.544184 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 471.750s; elapsed, 136.868s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 459.790s; elapsed, 136.847s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1062.710s; elapsed, 136.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 145.080s; elapsed, 136.836s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 459.790s; elapsed, 136.847s, #calls: 1', 'TOTAL : CPU, 459.790s; elapsed, 136.847s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 471.750s; elapsed, 136.868s, #calls: 1', 'TOTAL : CPU, 471.750s; elapsed, 136.868s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1062.710s; elapsed, 136.832s, #calls: 1', 'TOTAL : CPU, 1062.710s; elapsed, 136.832s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 856.740s; elapsed, 110.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.720s; elapsed, 0.240s, #calls: 1', 'time for EIGEN_LDLT: CPU, 442.120s; elapsed, 441.943s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 145.080s; elapsed, 136.836s, #calls: 1', 'TOTAL : CPU, 1444.660s; elapsed, 689.173s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 5040.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 161.630s; elapsed, 161.779s', 'individual call time for EIGEN_LDLT: CPU, 312.290s; elapsed, 312.512s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.312s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.39223', '# - matching time = 0.148616', '# - symmetrization time = 0.0774398', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.119725', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 124.784', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.53864e+11', '# - flop rate = 5.75295 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.26672e+11', '# --------------------------------------------', '# total = 7.26672e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.05517e-11\trel.res = 1.9251e-15\tbw.error = 6.12855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.218756', '# - total flops = 2.86208e+08, min = 226725, max = 1.89071e+08', '# - flop rate = 1.30834 GFlop/s', '# - bytes moved = 434.975 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.9884 GByte/s', '# - solve arithmetic intensity = 0.657987 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 126.810s; elapsed, 126.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 126.770s; elapsed, 126.878s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 126.700s; elapsed, 126.878s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 126.410s; elapsed, 126.860s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 126.550s; elapsed, 126.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 126.800s; elapsed, 126.864s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 126.790s; elapsed, 126.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 126.860s; elapsed, 126.873s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 126.800s; elapsed, 126.864s, #calls: 1', 'TOTAL : CPU, 126.800s; elapsed, 126.864s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 126.550s; elapsed, 126.855s, #calls: 1', 'TOTAL : CPU, 126.550s; elapsed, 126.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 126.770s; elapsed, 126.878s, #calls: 1', 'TOTAL : CPU, 126.770s; elapsed, 126.878s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 126.810s; elapsed, 126.877s, #calls: 1', 'TOTAL : CPU, 126.810s; elapsed, 126.877s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 126.790s; elapsed, 126.855s, #calls: 1', 'TOTAL : CPU, 126.790s; elapsed, 126.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 126.410s; elapsed, 126.860s, #calls: 1', 'TOTAL : CPU, 126.410s; elapsed, 126.860s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 126.700s; elapsed, 126.878s, #calls: 1', 'TOTAL : CPU, 126.700s; elapsed, 126.878s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 161.630s; elapsed, 161.779s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.312s, #calls: 1', 'time for EIGEN_LDLT: CPU, 312.290s; elapsed, 312.512s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 126.860s; elapsed, 126.873s, #calls: 1', 'TOTAL : CPU, 601.090s; elapsed, 601.476s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 5040.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 339.200s; elapsed, 175.051s', 'individual call time for EIGEN_LDLT: CPU, 323.380s; elapsed, 323.554s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.346s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.2369', '# - matching time = 0.140199', '# - symmetrization time = 0.0702751', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.142358', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 111.265', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.53864e+11', '# - flop rate = 6.45194 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.26672e+11', '# --------------------------------------------', '# total = 7.26672e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.04477e-11\trel.res = 1.92261e-15\tbw.error = 6.12855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.192233', '# - total flops = 2.86208e+08, min = 226725, max = 1.89071e+08', '# - flop rate = 1.48886 GFlop/s', '# - bytes moved = 434.988 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26282 GByte/s', '# - solve arithmetic intensity = 0.657967 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 157.940s; elapsed, 113.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 140.660s; elapsed, 113.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 123.230s; elapsed, 113.181s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 218.550s; elapsed, 113.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 121.770s; elapsed, 113.165s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 121.210s; elapsed, 113.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 152.840s; elapsed, 113.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 114.160s; elapsed, 113.174s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 121.770s; elapsed, 113.165s, #calls: 1', 'TOTAL : CPU, 121.770s; elapsed, 113.165s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 157.940s; elapsed, 113.179s, #calls: 1', 'TOTAL : CPU, 157.940s; elapsed, 113.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 152.840s; elapsed, 113.158s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 152.840s; elapsed, 113.158s', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 123.230s; elapsed, 113.181s, #calls: 1', 'TOTAL : CPU, 123.230s; elapsed, 113.181s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 121.210s; elapsed, 113.158s, #calls: 1', 'TOTAL : CPU, 121.210s; elapsed, 113.158s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 140.660s; elapsed, 113.179s, #calls: 1', 'TOTAL : CPU, 140.660s; elapsed, 113.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 218.550s; elapsed, 113.163s, #calls: 1', 'TOTAL : CPU, 218.550s; elapsed, 113.163s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 339.200s; elapsed, 175.051s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.346s, #calls: 1', 'time for EIGEN_LDLT: CPU, 323.380s; elapsed, 323.554s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 114.160s; elapsed, 113.174s, #calls: 1', 'TOTAL : CPU, 777.290s; elapsed, 612.125s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 5040.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 436.260s; elapsed, 113.660s', 'individual call time for EIGEN_LDLT: CPU, 337.400s; elapsed, 337.363s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.294s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.37134', '# - matching time = 0.156719', '# - symmetrization time = 0.0810502', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.161558', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 117.501', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.53864e+11', '# - flop rate = 6.10952 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.26672e+11', '# --------------------------------------------', '# total = 7.26672e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.16491e-11\trel.res = 1.95132e-15\tbw.error = 6.12855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.265008', '# - total flops = 2.86208e+08, min = 226725, max = 1.89071e+08', '# - flop rate = 1.08 GFlop/s', '# - bytes moved = 435.006 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.64148 GByte/s', '# - solve arithmetic intensity = 0.65794 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 190.160s; elapsed, 119.697s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 153.190s; elapsed, 119.697s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 237.680s; elapsed, 119.696s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 342.780s; elapsed, 119.670s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 148.250s; elapsed, 119.684s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 146.100s; elapsed, 119.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 455.700s; elapsed, 119.680s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 123.020s; elapsed, 119.691s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 146.100s; elapsed, 119.677s, #calls: 1', 'TOTAL : CPU, 146.100s; elapsed, 119.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 190.160s; elapsed, 119.697s, #calls: 1', 'TOTAL : CPU, 190.160s; elapsed, 119.697s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 342.780s; elapsed, 119.670s, #calls: 1', 'TOTAL : CPU, 342.780s; elapsed, 119.670s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 148.250s; elapsed, 119.684s, #calls: 1', 'TOTAL : CPU, 148.250s; elapsed, 119.684s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 153.190s; elapsed, 119.697s, #calls: 1', 'TOTAL : CPU, 153.190s; elapsed, 119.697s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 237.680s; elapsed, 119.696s, #calls: 1', 'TOTAL : CPU, 237.680s; elapsed, 119.696s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 455.700s; elapsed, 119.680s, #calls: 1', 'TOTAL : CPU, 455.700s; elapsed, 119.680s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 436.260s; elapsed, 113.660s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.294s, #calls: 1', 'time for EIGEN_LDLT: CPU, 337.400s; elapsed, 337.363s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 123.020s; elapsed, 119.691s, #calls: 1', 'TOTAL : CPU, 897.300s; elapsed, 571.008s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 2520.0625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 240.490s; elapsed, 241.224s', 'individual call time for EIGEN_LDLT: CPU, 358.420s; elapsed, 358.678s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.345s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.37219', '# - matching time = 0.142351', '# - symmetrization time = 0.053071', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.131288', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 101.497', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.5344e+11', '# - flop rate = 7.07285 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.25118e+11', '# --------------------------------------------', '# total = 7.25118e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.05649e-11\trel.res = 1.68642e-15\tbw.error = 6.01998e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.137746', '# - total flops = 3.60242e+08, min = 214125, max = 1.8812e+08', '# - flop rate = 2.61527 GFlop/s', '# - bytes moved = 413.137 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.99927 GByte/s', '# - solve arithmetic intensity = 0.871968 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 103.330s; elapsed, 103.428s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 103.280s; elapsed, 103.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 103.330s; elapsed, 103.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 103.290s; elapsed, 103.417s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 103.300s; elapsed, 103.427s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 103.240s; elapsed, 103.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 103.280s; elapsed, 103.427s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 103.290s; elapsed, 103.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 102.960s; elapsed, 103.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 103.320s; elapsed, 103.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 103.310s; elapsed, 103.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 103.340s; elapsed, 103.416s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 103.360s; elapsed, 103.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 103.100s; elapsed, 103.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 103.350s; elapsed, 103.414s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 103.390s; elapsed, 103.427s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 103.240s; elapsed, 103.426s, #calls: 1', 'TOTAL : CPU, 103.240s; elapsed, 103.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 103.280s; elapsed, 103.426s, #calls: 1', 'TOTAL : CPU, 103.280s; elapsed, 103.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 103.340s; elapsed, 103.416s, #calls: 1', 'TOTAL : CPU, 103.340s; elapsed, 103.416s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 103.330s; elapsed, 103.419s, #calls: 1', 'TOTAL : CPU, 103.330s; elapsed, 103.419s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 103.320s; elapsed, 103.419s, #calls: 1', 'TOTAL : CPU, 103.320s; elapsed, 103.419s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 103.310s; elapsed, 103.422s, #calls: 1', 'TOTAL : CPU, 103.310s; elapsed, 103.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 103.280s; elapsed, 103.427s, #calls: 1', 'TOTAL : CPU, 103.280s; elapsed, 103.427s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 103.350s; elapsed, 103.414s, #calls: 1', 'TOTAL : CPU, 103.350s; elapsed, 103.414s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 103.330s; elapsed, 103.428s, #calls: 1', 'TOTAL : CPU, 103.330s; elapsed, 103.428s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 103.290s; elapsed, 103.417s, #calls: 1', 'TOTAL : CPU, 103.290s; elapsed, 103.417s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 103.300s; elapsed, 103.427s, #calls: 1', 'TOTAL : CPU, 103.300s; elapsed, 103.427s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 102.960s; elapsed, 103.419s, #calls: 1', 'TOTAL : CPU, 102.960s; elapsed, 103.419s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 103.290s; elapsed, 103.426s, #calls: 1', 'TOTAL : CPU, 103.290s; elapsed, 103.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 103.100s; elapsed, 103.418s, #calls: 1', 'TOTAL : CPU, 103.100s; elapsed, 103.418s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 103.360s; elapsed, 103.415s, #calls: 1', 'TOTAL : CPU, 103.360s; elapsed, 103.415s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 240.490s; elapsed, 241.224s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.345s, #calls: 1', 'time for EIGEN_LDLT: CPU, 358.420s; elapsed, 358.678s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 103.390s; elapsed, 103.427s, #calls: 1', 'TOTAL : CPU, 702.640s; elapsed, 703.674s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '40321 2520.0625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 273.420s; elapsed, 140.993s', 'individual call time for EIGEN_LDLT: CPU, 335.590s; elapsed, 335.777s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.313s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 40,321', '# - number of nonzeros = 1,216,271', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,239', '# - number of levels = 129', '# - nd time = 1.34613', '# - matching time = 0.155253', '# - symmetrization time = 0.061341', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,239', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.15462', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.899 MB', '# - factor time = 111.341', '# - factor nonzeros = 112,987,369', '# - factor memory = 903.899 MB', '# - total flops = 7.17875e+11, min = 222688, max = 5.5344e+11', '# - flop rate = 6.44755 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.25118e+11', '# --------------------------------------------', '# total = 7.25118e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.04926e-11\trel.res = 1.68469e-15\tbw.error = 6.01998e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.151088', '# - total flops = 3.60242e+08, min = 214125, max = 1.8812e+08', '# - flop rate = 2.38432 GFlop/s', '# - bytes moved = 413.148 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.73448 GByte/s', '# - solve arithmetic intensity = 0.871946 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 121.430s; elapsed, 113.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 120.690s; elapsed, 113.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 138.460s; elapsed, 113.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 122.760s; elapsed, 113.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 156.400s; elapsed, 113.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 121.250s; elapsed, 113.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 138.310s; elapsed, 113.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 120.780s; elapsed, 113.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 119.900s; elapsed, 113.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 119.070s; elapsed, 113.303s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 183.730s; elapsed, 113.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 121.710s; elapsed, 113.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 120.490s; elapsed, 113.307s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 119.710s; elapsed, 113.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 215.650s; elapsed, 113.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 114.240s; elapsed, 113.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 121.250s; elapsed, 113.309s, #calls: 1', 'TOTAL : CPU, 121.250s; elapsed, 113.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 183.730s; elapsed, 113.305s, #calls: 1', 'TOTAL : CPU, 183.730s; elapsed, 113.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 156.400s; elapsed, 113.316s, #calls: 1', 'TOTAL : CPU, 156.400s; elapsed, 113.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 120.780s; elapsed, 113.309s, #calls: 1', 'TOTAL : CPU, 120.780s; elapsed, 113.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 121.430s; elapsed, 113.315s, #calls: 1', 'TOTAL : CPU, 121.430s; elapsed, 113.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 119.900s; elapsed, 113.305s, #calls: 1', 'TOTAL : CPU, 119.900s; elapsed, 113.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 121.710s; elapsed, 113.305s, #calls: 1', 'TOTAL : CPU, 121.710s; elapsed, 113.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 120.690s; elapsed, 113.317s, #calls: 1', 'TOTAL : CPU, 120.690s; elapsed, 113.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 119.070s; elapsed, 113.303s, #calls: 1', 'TOTAL : CPU, 119.070s; elapsed, 113.303s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 138.460s; elapsed, 113.315s, #calls: 1', 'TOTAL : CPU, 138.460s; elapsed, 113.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 119.710s; elapsed, 113.308s, #calls: 1', 'TOTAL : CPU, 119.710s; elapsed, 113.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 138.310s; elapsed, 113.316s, #calls: 1', 'TOTAL : CPU, 138.310s; elapsed, 113.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 122.760s; elapsed, 113.315s, #calls: 1', 'TOTAL : CPU, 122.760s; elapsed, 113.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 215.650s; elapsed, 113.308s, #calls: 1', 'TOTAL : CPU, 215.650s; elapsed, 113.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 120.490s; elapsed, 113.307s, #calls: 1', 'TOTAL : CPU, 120.490s; elapsed, 113.307s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 273.420s; elapsed, 140.993s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.313s, #calls: 1', 'time for EIGEN_LDLT: CPU, 335.590s; elapsed, 335.777s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 114.240s; elapsed, 113.317s, #calls: 1', 'TOTAL : CPU, 723.700s; elapsed, 590.401s']
Data Set Size:=1k
Skipping 1k
Data Set Size:=5k
Skipping 5k
Data Set Size:=32k
Skipping 32k
import json, cPickle
with open("OMP_SOLVER_TIME_10k_MPI.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_10k_MPI.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_10k_MPI.pickle', 'rb'))
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_10k_omp1_params','_'):str_out[k]})
import numpy as np
u_dat = {}
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = np.empty(shape=(5,5))
same_t.strum_mpi_ptscotch = np.empty(shape=(5,5))
same_t.eig_ldlt = np.empty(shape=(5,5))
same_t.eig_bicgstab = np.empty(shape=(5,5))
same_t.strum_scotch_a[:] = np.nan
same_t.strum_mpi_ptscotch[:] = np.nan
same_t.eig_ldlt[:] = np.nan
same_t.eig_bicgstab[:] = np.nan
for m_idx,m in enumerate(mpi_proc_list):
for t_idx,t in enumerate(threads_list):
if m*t <=32:
t_STRUMPACK_OMP = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'STRUMPACK_OMP' in s]
t_STRUMPACK_MPI_DIST_RANK = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'STRUMPACK_MPI_DIST_RANK' in s]
t_EIGEN_LDLT = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'EIGEN_LDLT' in s]
t_EIGEN_BICGSTAB = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'EIGEN_BICGSTAB' in s]
if m != 1:
same_t.strum_mpi_ptscotch[m_idx,t_idx] = t_STRUMPACK_MPI_DIST_RANK[-1].rsplit(',')[-2].strip()[0:-1]
same_t.strum_scotch_a[m_idx,t_idx] = t_STRUMPACK_OMP[-1].rsplit(',')[-2].strip()[0:-1]
same_t.eig_bicgstab[m_idx,t_idx] = t_EIGEN_BICGSTAB[-1].rsplit(',')[-2].strip()[0:-1]
same_t.eig_ldlt[m_idx,t_idx] = t_EIGEN_LDLT[-1].rsplit(',')[-2].strip()[0:-1]
u_dat.update({u:same_t})
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SA = "STR_SA_%s"%u
str_S_MPI = "STR_MPI_%s"%u
df_dict = { str_E_LDLT:same_t.eig_ldlt,
str_E_BICGSTAB:same_t.eig_bicgstab,
str_S_SA:same_t.strum_scotch_a,
str_S_MPI:same_t.strum_mpi_ptscotch
}
df_dict
With the data correctly parsed, we can plot the performance of each solver with $t\left(\textrm{MPI}, \textrm{OMP}\right)$. Note that the MPI solver does not operate with a single thread, and so will not have data in this row. Subsequently, the relative performance of the OpenMP STRUMPACK backend to the MPI backend is shown. While it would be best compared for the same upp er limit of threads*processes, it can be sufficient to see scalability across the board to identify MPI-enabled sweet-spots. Additionally, as no MPI benefit exists for the other solvers, the launching of processes and overheads can slow performance overal. Thus, it is best to examine the single process data for the non MPI-enabled data.
STRUMPACK MPI
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('10k strum_mpi_ptscotch', size=16)
ax = []
for i, u in enumerate(uniq_ref):
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(u_dat[u].strum_mpi_ptscotch)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
STRUMPACK OpenMP
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('10k strum_scotch_a', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].strum_scotch_a
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
plt.show()
 |
EIGEN LDLT
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('10k eig_ldlt', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].eig_ldlt
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
EIGEN BICGSTAB
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('10k eig_bicgstab', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].eig_bicgstab
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('10k strum_mpi_ptscotch / strum_scotch_a', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = np.divide(u_dat[u].strum_mpi_ptscotch , u_dat[u].strum_scotch_a)
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data,cmap='coolwarm', vmin=0.25, vmax=1.75)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
